Relocation module and methods for surgical equipment

ABSTRACT

Module for housing electronic and electromechanical medical equipment including a portable digital camera and processing circuitry with machine vision and machine learning software for automatically documenting healthcare events and healthcare equipment operations in the electronic health record.

This application is a continuation of U.S. application Ser. No.17/528,832 filed Nov. 17, 2021, which is a continuation-in-part of U.S.application Ser. No. 17/376,469 filed Jul. 15, 2021, now issued as U.S.Pat. No. 11,219,570, which is a continuation-in-part of U.S. applicationSer. No. 17/199,722 filed Mar. 12, 2021, now issued as U.S. Pat. No.11,173,089, which is a continuation of U.S. application Ser. No.17/092,681, filed Nov. 9, 2020, now issued as U.S. Pat. No. 10,993,865,which is a continuation of U.S. application Ser. No. 16/879,406, filedMay 20, 2020, now issued as U.S. Pat. No. 10,869,800, which is acontinuation-in-part of U.S. application Ser. No. 16/601,924, filed Oct.15, 2019, now issued as U.S. Pat. No. 10,702,436, which is acontinuation of U.S. application Ser. No. 16/593,033, filed Oct. 4,2019, now issued as U.S. Pat. No. 10,653,577, which is a continuation ofU.S. application Ser. No. 16/364,884, filed Mar. 26, 2019, now issued asU.S. Pat. No. 10,507,153, which claims the benefit of priority to U.S.Provisional Patent Application 62/782,901, filed Dec. 20, 2018. U.S.application Ser. No. 16/364,884, filed Mar. 26, 2019, now issued as U.S.Pat. No. 10,507,153 is also a continuation-in-part of U.S. applicationSer. No. 15/935,524, filed Mar. 26, 2018, now issued as U.S. Pat. No.10,512,191.

U.S. application Ser. No. 17/376,469 is also a continuation-in-part ofU.S. application Ser. No. 17/245,942, filed Apr. 30, 2021, which is acontinuation-in-part of U.S. application Ser. No. 17/167,681, filed Feb.4, 2021, now issued as U.S. Pat. No. 11,160,710, which is acontinuation-in-part of U.S. application Ser. No. 17/092,681, filed Nov.9 2020, now issued as U.S. Pat. No. 10,993,865, which is a continuationof U.S. application Ser. No. 16/879,406, filed May 20, 2020, now issuedas U.S. Pat. No. 10,869,800.

The disclosures of each of these applications is incorporated herein byreference in its entirety.

TECHNICAL FIELD

This document pertains generally, but not by way of limitation, tosystems and methods for improving safety in operating rooms andhospitals. In particular, the systems and methods described herein mayinclude but are not limited to, anesthetic, surgical and medicalequipment storage and operational data capture, automated anesthetic andpatient monitoring data capture and electronic record input.

BACKGROUND

Anesthesia monitors and equipment as well as surgical equipment havebeen invented, developed and sporadically introduced into surgicalpractice over more than a century. This equipment is made by a widevariety of companies who have no incentive to coordinate with oneanother to create the most efficient operating room. Equipmentthroughout the operating room has been placed in one location oranother, generally without a plan and then decades later, is stillsitting in that unplanned location.

Over the past 20 years, there has been a gradual movement to replacingpaper anesthetic records with electronic anesthetic records (EAR). Thedigital electronic data outputs of the patient's physiologic monitorshave been relatively easy to input into the EAR. However, the identity,dosing and timing of IV and inhaled drug administration, IV fluidadministration, oxygen and ventilation gas administration and anestheticevents such as intubation have required manual input to the EAR by wayof a computer keyboard and mouse. Blood, fluid and urine outputs havealso required manual input to the EAR by way of a computer keyboard andmouse. The surgical equipment scattered around the operating room eitherdoes not produce a digital output that could memorialize the equipment'soperation to the electronic record, that output is not automaticallycaptured, or the output is not provided in a way that providesmeaningful context.

Carefully observing the patient in various conditions and situationsincluding surgery has been an important source medical information forcenturies. However, in this age of electronic monitoring, patientobservation by the healthcare provider is becoming a lost art that isinfrequently done and if it is done it may not be entered into therecord so the information is lost.

Most electronic medical equipment, especially equipment constitutingdose events, do not produce an electronic data record documenting theequipment's operating parameters. A complete electronic health record(EHR) requires instant documentation of all dose events in order toallow meaningful clinical decision support (CDS) and AI analysis of doseand response events.

The concept of “garbage in, garbage out” is of the utmost importance formedical algorithms trained on healthcare datasets. The majority of datacategories in the acute care setting are manually inputted, basically adigitized paper record. Manually inputted data is sporadic and prone toerrors and omissions. It is also not time-stamped for temporalcorrelations. The result is that when those data are aggregated into a“big data” database, the data is incomplete, inconsistent, missing andoften unusable. Further, clinicians hate the data entry part of theirjobs, seeing it as time consuming and distracting from patient care.Several studies have linked manual data entry to the electronic recordas a significant contributor to physician burnout.

SUMMARY

In some examples, the automated data consolidation module of thisdisclosure minimizes “garbage in” of healthcare “big data” by automatingthe data input process.

As a result of the current medical practices described, the majority ofthe input to the electronic anesthetic and medical records has been thedata from the vital signs monitors, recording the patient's “response.”The “dose” events (things that are given or done to the patient leadingto the “response”) are manually entered into the record, resulting inmistakes, omissions and no temporal correlation between the dose andresponse. The incomplete and inaccurate records make any analysis withartificial intelligence and machine learning software problematic,either for that individual patient or for “big data” analysis ofpopulations of patients.

This document pertains generally to systems and methods for improvingsafety for patients receiving intravenous (IV) medications and fluids,by avoiding medication or fluid errors and documenting theadministration. This document pertains generally, but not by way oflimitation, to systems and methods for constructing granular(beat-by-beat) anesthetic, surgical and patient records that includeboth “dose” events (things that are given or done to the patient) and“response” events (inputs from electronic monitors, measurement devicesand machine vision “observations”). The dose and response events areprecisely temporally related and recorded in the patient's electronicrecord and may be pooled with the records of other patients in adatabase that can be analyzed with artificial intelligence and machinelearning software.

Illustrative examples of an automated data consolidation module thatsystematizes surgical safety for patients and OR personnel. In someexamples, this automated data consolidation module is designed to housenearly all of the operating room patient monitors and support equipment.Even dissimilar types of equipment that are normally kept separate fromone another. In some examples, this unique automated data consolidationmodule is specially designed to fit next to and under the arm-board ofthe surgical table—a location traditionally occupied by an IV pole. Forthe past 100 years, this location has been a wasted “no-man's land”between the anesthesia and surgical sides of the operating room. Inreality, the unique space next to and under the arm-board, is truly the“prime real estate” of the entire operating room: it is immediatelyadjacent the patient for optimal monitoring while simultaneouslymaintaining observation of the patient and surgical procedure; equipmentcontrols can be conveniently accessed by both the anesthesia andsurgical staff; short cables and hoses are adequate to reach thepatient; and it is uniquely accessible from both the anesthesia andsurgical sides of the anesthesia screen. The unique space next to andunder the arm-board is the only location in the entire operating roomwhere cables, cords and hoses from both the anesthesia side and thesterile surgical field side, do not need to traverse the floor or eventouch the floor in order to connect to their respective monitor orpatient support equipment—truly a remarkable location that has beenwasted by conventional systems.

In some examples, an illustrative automated data consolidation modulecan house both anesthesia related and non-anesthesia related equipment.In some examples, the illustrative relocation module can house a varietyof non-proprietary OR equipment such as patient vital sign monitors,electro-surgical generators, anesthesia machines and mechanicalventilators. In some examples, the automated data consolidation moduleis designed to also house newer proprietary safety equipment such as:air-free electric patient warming, surgical smoke evacuation, wastealcohol and oxygen evacuation, evacuation of the flow-boundarydead-zones that cause disruption of the OR ventilation and theevacuation and processing of waste heat and air discharged from ORequipment. In some examples, this automated data consolidation modulemay also house dissimilar equipment (e.g., unrelated to anesthesiamonitoring) such as: air mattress controls and air pumps; sequentialcompression legging controls and air pumps; capacitive couplingelectrosurgical grounding; RFID counting and detection of surgicalsponges; the waste blood and fluid disposal systems; and “hover”mattress inflators. Any of these devices may be stored in the automateddata consolidation module together with (or without) anesthesiaequipment.

In some examples, the automated data consolidation module is aspecialized and optimally shaped rack for holding and protecting thepatient monitors and other electronic and electromechanical surgicalequipment, in a unique location. A location that is very different fromjust setting anesthesia monitors on top of the anesthesia machine andscattering other equipment across the floor of the operating room. Theautomated data consolidation module may be used anywhere throughout thehospital or long term care settings.

The various pieces of electronic and electromechanical equipment housedwithin the automated data consolidation module disclosed herein canproduce relatively large amounts of waste heat. In some examples, theautomated data consolidation module may include a waste heat managementsystem to safely dispose of the waste heat created by the electronic andelectromechanical equipment housed within the automated dataconsolidation module.

It would be difficult or even impossible to manage the uncontained wasteheat produced by electronic and electromechanical equipment mounted on asimple open rack because it can escape in any direction. In someexamples, the module can include a “cowling” covering some or all of theouter surface. The cowling not only protects the equipment fromaccidental fluid damage but also confines the waste heat from theelectronic and electromechanical equipment mounted within the module, tothe inside of the module and cowling. In some examples, the confinedwaste heat can then be safely managed.

In some examples, the cowling cover of the automated data consolidationmodule can form or support a waste heat management system. In someexamples, the cowling can be provided on an inner surface of thehousing. In some examples, the cowling can be described as aninsulation. In some examples, the housing can include other types ofinsulation from heat and/or water. Any suitable type of insulatedhousing suitable for use in a surgical field can be provided.

In some examples, the automated data consolidation module of the instantinvention may also contain the components of the anesthesia gas machine.So-called “gas machines” are relatively simple assortments of piping,valves, flow meters, vaporizers and a ventilator. These could be locatedwithin the automated data consolidation module or attached to theautomated data consolidation module for further consolidation ofequipment and for improved access to the patient.

In some examples, locating the anesthesia machine in or on the automateddata consolidation module allows direct access for and sensors andmonitors related to the anesthesia machine, to input data to theelectronic anesthetic record being recorded by equipment in theautomated dose/response record system.

In some examples, the collection canisters for waste fluid and blood maybe conveniently mounted on the module.

In some examples, the controls and display screens for the surgicalequipment housed in the automated data consolidation module may bewirelessly connected to a portable display screen such as an iPad or“smart tablet,” for convenient access by the nurse anywhere in the room.A remote display screen can also allow remote supervision andconsultation.

In some examples the automated data consolidation module may be locatednext to the patient's bed in the ICU, ER, on the ward or in long termcare. While most of the data collected by the automated dataconsolidation module will occur in the acute care setting, it should beunderstood that the automated data consolidation module concept forautomatically collecting and consolidating data from a wide variety ofdata sources including monitors and other medical equipment, can beapplied throughout the healthcare delivery system.

Doctors and nurses dislike record keeping and the switch to theelectronic record has made the act of record keeping more difficult andtime consuming. Entering the electronic record into the computersometime after the event occurred and the case has settled down, is notonly distracting from patient care but leads to inaccurate records. Handentered records also bypass the opportunity for the computer to add topatient safety by checking drug identities, dosages, side effects,allergies and alerting the clinician to potential problems or evenphysically stopping the drug administration. Manually entered recordsare not useful for managing drug inventories because a given medicationadministration is not tied to a specific drug bottle or syringe.Finally, the computer mouse and keyboards have been shown to becontaminated by a wide variety of infective organisms and are virtuallyimpossible to clean. Automatic anesthetic data entry to the EAR wouldimprove patient safety, improve clinician job satisfaction and improveOR inventory management.

In general, doctors and nurses are not interested in replacingthemselves and their jobs with automated drug delivery or automatedanesthesia systems. However, they may be more open to automated recordkeeping. The challenge with automated record keeping is automating thedata input that documents the numerous activities, anesthesia relatedevents, fluid, gas and medication administration that constitute ananesthetic experience or another medical situation.

The second challenge in implementing an automated electronic anestheticrecord (EAR) or automated electronic medical record (EMR) is to force aslittle change in routine as possible onto the anesthesiologist and otherclinicians using this system. Anesthesiologists and surgeons arenotoriously tradition-bound and resistant to any changes in their “triedand true” way of doing things. Therefore, a successful automated EARmust interact seamlessly with current anesthesia practices and operatingroom workflow without causing any disruptions.

In some examples, the automated data consolidation module of thisdisclosure includes a system for automatically measuring and recordingthe administration of IV medications and fluids. The system forautomatically measuring and recording the administration of IVmedications and fluids can include one or more sensors, such as one ormore of a barcode reader and an RFID interrogator for accuratelyidentifying a medication or fluid for IV administration.

In some examples, the system for automatically measuring and recordingthe administration of IV medications and fluids can also include one ormore digital cameras with machine vision software (“machine vision”) foraccurately measuring the volume of medication administered from asyringe or fluid administered from an IV bag through a drip chamber intoan IV tubing. The digital cameras with machine vision softwareessentially duplicate the clinician's vision of an activity, injectionof a drug from a syringe for example, without interfering in the normalactivity and yet allows automatic recording of the activity in the EAR.The machine vision software can include one or more machine-readablemediums that when implemented on hardware processing circuitry of thesystem or in electrical communication with the system, can perform thefunctions described herein.

In some examples, the automated data consolidation module of thisdisclosure uses machine vision to unobtrusively “observe” the flow rateof the ventilation gas flow meters and inhaled anesthetic vaporizers.

In some examples, the automated data consolidation module of thisdisclosure captures input data from the blood and fluid collection andurine output collection systems of this disclosure.

In some examples, the automated data consolidation module of thisdisclosure lets the computer (e.g., a processor and memory forperforming instructions) add to patient safety by checking drugidentities, dosages, side effects, allergies, the patients' medicalhistory and vital signs and alerting the clinician to potential problemsor even physically stopping the drug administration. In some examples,the automated data consolidation module of this disclosure eliminatesmedication errors by checking the drug to be injected against thephysician's medication orders before the injection can occur. In someexamples, the automated data consolidation module of this disclosure isuseful for managing drug inventories because a given medicationadministration is tied to a specific drug bottle or syringe.

In some examples, the automated data consolidation module of thisdisclosure may also automatically record and display many otherfunctions including but not limited to: IV fluid administration,medication infusions, the patient's vital signs, urine output, bloodloss, ventilator settings, inspired gases, electrosurgical settings,pneumoperitoneum insufflation settings, RFID surgical sponge counts,surgical information and video, dialysis or other medical procedureinformation and patient activity.

“Dose/response” is one of the most basic of all medical processes. Sincethe beginning of medical practice, both the art and science of medicinehave relied on giving something to the patient (a medicine for example)or doing something to the patient (mechanical ventilation or surgery forexample)—the “dose”, and then observing the patient's “response.” Theproblem now seen with electronic records is that the only data that istimely recorded is the “response” data provided by the physiologicmonitors. Even that response data is frequently not recordedbeat-by-beat but rather intermittently recorded every 5 minutes or 30minutes or 4 hours for example. All of the “dose” data is entered intothe electronic record by hand and therefore is prone to mistakes,omissions and unknown timing. Therefore, with current EMRs, the dose andresponse data cannot be temporally correlated with any accuracy, vastlyreducing the analytical and predictive value of the electronic databaseand record.

In some examples, the automated data consolidation module of thisdisclosure includes systems and methods for constructing granular(beat-by-beat, second-by-second) anesthetic, surgical and patientrecords that include both “dose” events—the things that are given ordone to the patient (inputs from medication injection and fluidmonitors, various support equipment and machine vision “observations”for example) and “response” events (inputs from electronic monitors,measurement devices and machine vision “observations” for example). Insome examples, the invention of this disclosure automatically entersboth dose and response events into the electronic record. In someexamples, the invention of this disclosure automatically enters bothdose and response events into the electronic record and temporallycorrelates the dose and response events, such as but not limited to,when they are recorded. In some examples, the automatically entered,temporally correlated dose and response events in the patient'selectronic record may be analyzed by artificial intelligence (AI) and/ormachine learning (ML) software stored in a memory of a storage deviceelectrically coupled to the processing circuitry of the module forimmediate advice, alerts and feedback to the clinician. In someexamples, the automatically entered, temporally correlated dose andresponse events in the patient's electronic record may be pooled withthe records of other patients in a database that can be analyzed withartificial intelligence and machine learning software.

Machine Learning (ML) is an application that provides computer systemsthe ability to perform tasks, without explicitly being programmed, bymaking inferences based on patterns found in the analysis of data.Machine learning explores the study and construction of algorithms(e.g., tools), that may learn from existing data and make predictionsabout new data. Such machine-learning algorithms operate by building anML model from example training data, in order to make data-drivenpredictions or decisions expressed as outputs or assessments. Theprinciples presented herein may be applied using any suitablemachine-learning tools.

In some examples, the AI or ML software can compare dose and responseevents from different periods of time for the same patient to learn andidentify the particular patient's responses. In some examples themachine learning software can be trained to identify a patient'sresponses using training obtained from a plurality of patients that mayinclude or not include the patient being monitored. Any suitable AI orML can be implemented to interpret the data generated by the module.Current methods of obtaining data in medical settings cannot generate,store or aggregate such data for analysis using AI or ML. Thus, thedose-response systems described herein provide a technical solution to atechnical problem.

Machine vision cameras and software are very good at measuringdistances, movements, sizes, looking for defects, fluid levels, precisecolors and many other quality measurements of manufactured products.Machine vision cameras and software can also be “taught” through AI andML to analyze complex and rapidly evolving scenes, such as those infront of a car driving down the road.

In some examples, the automated data consolidation module of thisdisclosure includes novel systems and methods for using portable machinevision cameras and software to “observe” the operating parameters ofvarious dose event and other equipment. In some examples, the automateddata consolidation module of this disclosure includes novel systems andmethods for using portable machine vision cameras and software to“observe” the response parameters of various response event equipment.In some examples, the portable machine vision cameras may include: laserpointers to aid in aiming and coded labels on the items or devices to beidentified to aid in ML identification. In some examples, the laserpointers and coded labels help to structure the scene so that ML can besimplified.

In some examples, the automated data consolidation module of thisdisclosure includes novel systems and methods for using machine visioncameras and software to “observe” the patient. If the patient is insurgery, the patient's head may be the focus of the observation. In someexamples, during surgery the machine vision cameras and software may be“looking” for dose events including but not limited to mask ventilationor endotracheal intubation. In some examples, during surgery the machinevision cameras and software may be “looking” for response eventsincluding but not limited to grimacing or tearing or coughing or changesin skin color.

If the patient is on the ward or in the nursing home or other long-termcare facility, the whole patient may be the focus of the observation. Insome examples, if the patient is on the ward or in the nursing home orother long-term care facility the machine vision cameras, processingcircuitry and software may be configured to “look” for dose events(e.g., sense) including but not limited to repositioning the patient,suctioning the airway or assisting the patient out of bed or any othernursing procedure, eating and drinking. In some examples, if the patientis on the ward or in the nursing home or other long-term care facility(including at home) the machine vision cameras, processing circuitry andsoftware may be configured to “look” for response events (e.g., sense)including but not limited to restlessness or getting out of bed withoutassistance or coughing or breathing pattern. In some examples, thesystem can go beyond traditional physiologic monitors. Even physiologicresponse information such as pain may be detected by facial expressionanalysis.

In some examples, vital signs such as heart rate, respiration rate,blood oxygen saturation and temperature can be measured (e.g., sensed,monitored) remotely via camera-based methods. Vital signs can beextracted from the optical detection of blood-induced skin colorvariations—remote photoplethysmography (rPPG).

In some examples, the automated data consolidation module may allowremote viewing of the displayed patient information. In some examples,the remotely displayed patient information may be used for remotemedical supervision such as an anesthesiologist providing remotesupervision to a nurse anesthetist who is administering the anesthetic.In some examples, the remotely displayed patient information may be usedfor remote medical consultation. In some examples, the remotelydisplayed patient information may be used to document the involvement ofremote medical supervision or consultation for billing purposes.

In some examples, the automated data consolidation module allows rulesto be applied to the various medical equipment that is housed within themodule, mounted on the module, or is in electrical communication with orin wireless communication with the module. In some examples, the rulescan include one or more of the following: that all equipment producedata reflecting the equipment's operating parameters and sensor inputs,the data is produced in prescribed data formats, the data include allprescribed input record fields for that specific type of equipment, thedata is instantly and continuously provided.

In some examples, the automated data consolidation module includesprocessing circuitry and software that accept the data inputted from thevarious medical equipment. In some examples, the processing circuitryand software can translate data that is not inputted in the prescribedformat. In some examples, the processing circuitry and software can addtime stamps to the data to add a temporal context. In some examples, theprocessing circuitry and software can do data “filtering” in thepresence of large size data to discard information that is not usefulfor healthcare monitoring based on a defined criterion. This may includefor example, intermittently recording data that changes slowly such asthe patient's temperature, rather than continuously recording. In someexamples, the processing circuitry and software can do data “cleaning”such as normalization, noise reduction and missing data management.Sensor fusion is a technique that may be utilized to simultaneouslyanalyze data from multiple sensors, in order to detect erroneous datafrom a single sensor. In some examples, the processing circuitry andsoftware can be used in many other ways to cleanse, organize and preparethe input data.

In some examples, the processing circuitry and software execute “streamprocessing” for applications requiring real-time feedback. In someexamples, streaming data analytics in healthcare can be defined as asystematic use of continuous waveform and related medical recordinformation developed through applied analytical disciplines, to drivedecision making for the patient care.

In some examples, when the objective is to deliver data to a “big data”database, the data must be pooled. Data in the “raw” state needs to beprocessed or transformed. In a service-oriented architectural approach,the data may stay raw and services are used to call, retrieve andprocess the data. In the data warehousing approach, data from varioussources is aggregated and made ready for processing, although the datais not available in real-time. The steps of extract, transform, and load(ETL) can be used to cleans and ready data from diverse sources.

In some examples, with “big data” database data, the processingcircuitry and software may execute “batch processing,” analyzing andprocessing the data over a specified period of time. Batch processingaims to process a high volume of data by collecting and storing batchesto be analyzed in order to generate results. In some examples, theprocessing circuitry and software can serve as a “node” in batchcomputing, where big data is split into small pieces that aredistributed to multiple nodes in order to obtain intermediate results.Once data processing by nodes is terminated, outcomes will be aggregatedin order to generate the final results.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings, which are not necessarily drawn to scale, like numeralsmay describe similar components in different views. Like numerals havingdifferent letter suffixes may represent different instances of similarcomponents. The drawings illustrate generally, by way of example, butnot by way of limitation, various examples discussed in the presentdocument. Any combination of the features shown and described in thisdisclosure, including combinations of fewer or more features is withinthe content of this disclosure. Modules, systems and methods includingindividual features described herein, without combinations of featuresas shown in the examples (for the sake of brevity), are also within thescope of this disclosure.

FIG. 1 illustrates an isometric view of an example module including asystem for generating an automated electronic anesthetic record locatedproximate to a patient, in accordance with at least one example of thisdisclosure.

FIG. 2 illustrates an isometric view of an example module including asystem for generating an automated electronic anesthetic record locatedproximate to a patient, in accordance with at least one example of thisdisclosure.

FIG. 3 illustrates a plan view of an example of preloaded syringes thatcan be used with the system of FIGS. 1 and 2, in accordance with atleast one example of this disclosure.

FIG. 4 illustrates a side view of an example medication identificationand measurement system and a syringe that can be used with the system ofFIGS. 1 and 2, to monitor drug delivery, in accordance with at least oneexample of this disclosure.

FIG. 5 illustrates a cross-sectional view of the medicationidentification and measurement system and a syringe (not shown incross-section) of FIG. 4, taken along line 5-5, in accordance with atleast one example of this disclosure.

FIG. 6 illustrates a side view of a second example medicationidentification and measurement system and a syringe that can be usedwith the system of FIGS. 1 and 2, in accordance with at least oneexample of this disclosure.

FIG. 7 illustrates a cross-sectional view of the second example of amedication identification and measurement system and the syringe (notshown in cross-section) of FIG. 6, taken along line 7-7, in accordancewith at least one example of this disclosure.

FIG. 8 illustrates a side view of a third example of a medicationidentification and measurement system and a syringe that can be usedwith the system of FIGS. 1 and 2, in accordance with at least oneexample of this disclosure.

FIG. 9 illustrates a cross-sectional view of the third example of amedication identification and measurement system and the syringe (notshown in cross-section) of FIG. 8, taken along line 9-9, in accordancewith at least one example of this disclosure.

FIG. 10 illustrates an example injection port cassette that can be usedwith the system of FIGS. 1 and 2, as detailed in FIGS. 5, 7 and 9, inaccordance with at least one example of this disclosure.

FIG. 11 illustrates a plan view of an example of healthcare provider IDbadges that can be used with the system of FIGS. 1 and 2, in accordancewith at least one example of this disclosure.

FIG. 12 illustrates an isometric view of another example moduleincluding a system for generating an automated electronic anestheticrecord located proximate to a patient, in accordance with at least oneexample of this disclosure.

FIG. 13 illustrates an isometric view of another example moduleincluding a system for generating an automated electronic anestheticrecord located proximate to a patient, in accordance with at least oneexample of this disclosure.

FIG. 14 illustrates a side view of an example IV fluid identificationand measurement system that can be used with the systems of FIGS. 1 and2, and injection port cassette of FIG. 10, in accordance with at leastone example of this disclosure.

FIG. 15 illustrates generally an example of a block diagram of a machine(e.g., of module 101, 201) upon which any one or more of the techniques(e.g., methodologies) discussed herein may perform in accordance with atleast one example of this disclosure.

FIG. 16 is a flow chart illustrating a technique of IV fluididentification and measurement, in accordance with at least one exampleof this disclosure.

FIG. 17 is a second flow chart illustrating the technique of IV fluididentification and measurement, in accordance with at least one exampleof this disclosure.

FIG. 18 is a flow chart illustrating a technique of medicationidentification and measurement, in accordance with at least one exampleof this disclosure.

FIG. 19 is a second flow chart illustrating a technique of medicationidentification and measurement, in accordance with at least one exampleof this disclosure.

FIG. 20 is a flow chart illustrating a second technique of IV fluididentification and measurement including safety and security aspects, inaccordance with at least one example of this disclosure.

FIG. 21 is a second flow chart illustrating a second technique of IVfluid identification and measurement including safety and securityaspects, in accordance with at least one example of this disclosure.

FIG. 22 illustrates generally an example of a block diagram of vendingsystem and a medication delivery system of FIGS. 1-21 upon which any oneor more of the techniques (e.g., methodologies) discussed herein mayperform in accordance with at least one example of this disclosure.

FIG. 23 is a flow chart illustrating a technique 2300 for assessingphysiologic events, in accordance with at least one example of thisdisclosure.

FIG. 24 is a flow chart illustrating a technique 2400 for assessingphysiologic events, in accordance with at least one example of thisdisclosure.

FIG. 25 is a flow chart illustrating a technique 2500 for assessingphysiologic events, in accordance with at least one example of thisdisclosure.

FIG. 26 is a flow chart illustrating a technique 2600 for creating bigdata, in accordance with at least one example of this disclosure.

FIG. 27 illustrates a cross-sectional view of an example of machinevision counting of pills, in accordance with at least one example ofthis disclosure.

FIG. 28 illustrates a side view of an example of a portable digitalcamera, in accordance with at least one example of this disclosure.

FIG. 29 illustrates a front view of an item or device to be recorded, inaccordance with at least one example of this disclosure.

FIGS. 30A-D illustrates front views of items or devices to be recorded,in accordance with at least one example of this disclosure.

FIG. 31 illustrates a front view of an item or device to be recorded, inaccordance with at least one example of this disclosure.

FIG. 32 illustrates a top view of an example of a portable digitalcamera, in accordance with at least one example of this disclosure.

FIGS. 33A-C illustrates targets and QR codes that can be applied to theitems or devices to be recorded, in accordance with at least one exampleof this disclosure.

FIG. 34 illustrates a perspective view of an example of a head-updisplay (HUD), in accordance with at least one example of thisdisclosure.

FIG. 35 illustrates a perspective view of an example of a head-updisplay (HUD), in accordance with at least one example of thisdisclosure.

FIG. 36 is a flow chart illustrating a technique or process 3600 fordocumenting dose events with a portable digital camera, in accordancewith at least one example of this disclosure.

DETAILED DESCRIPTION

The following detailed description is exemplary in nature and is notintended to limit the scope, applicability, or configuration of theinvention in any way. Rather, the following description providespractical illustrations for implementing exemplary examples of thepresent invention. Examples of constructions, materials, dimensions, andmanufacturing processes are provided for selected elements, and allother elements employ that which is known to those of skill in the fieldof the invention. Those skilled in the art will recognize that many ofthe examples provided have suitable alternatives that can be utilized.

As described herein, operably coupled can include, but is not limitedto, any suitable coupling, such as a fluid (e.g., liquid, gas) coupling,an electrical coupling or a mechanical coupling that enables elementsdescribed herein to be coupled to each other and/or to operate togetherwith one another (e.g., function together).

“Dose/response” can be a useful medical tool. Dose/response involvesgiving something to the patient (a medicine for example) or doingsomething to the patient (mechanical ventilation or surgery forexample)—the “dose”, and then observing the patient's “response.”

Throughout this disclosure, the word “dose” or “dose event” meanssomething that was given to or done to the patient. In some examples,“dose events” involve giving something to the patient including but notlimited to: the injection or ingestion of medications, the infusion ofmedications, the infusion of IV fluids, inspired gases such as oxygen ornitrous oxide or volatile anesthetics or nitric oxide.

In some examples, “dose events” are things done to the patient,including but not limited to: mask ventilation, tracheal intubation,mechanical ventilation, pneumoperitoneum insufflation, patientpositioning and surgery.

In some examples, “dose events” are applications of electricity to thepatient, including but not limited to: train-of-four determination ofmuscle relaxation, nerve conduction studies, cardiac pacing,cardioversion, electrical stimulation of nerves and electroconvulsivetherapy. Many other “dose events” are anticipated.

Throughout this disclosure, the word “response” or “response event”means a physiologic, reflexive motor response or volitional motorresponse from the patient. In some examples, “response events” involvemeasuring a physiologic response of the patient measured by thephysiologic monitors, including but not limited to: an electrocardiogram(EKG), pulse oximetry, blood pressure, end tidal gases,electroencephalogram (EEG), bispectral index (BIS), laboratory bloodstudies, urine output, blood loss and pulmonary compliance.

In some examples, “response events” involve measuring a motor responseof the patient, including but not limited to: a train-of-four,grimacing, coughing, movements of many sorts and tearing, that may bemeasured by a machine vision camera and software.

FIG. 1 illustrates an isometric view of an example automated dataconsolidation module100 for generating an automated electronicanesthetic record (EAR) or electronic medical record (EMR) locatedproximate to a patient. Some aspects of FIG. 1 are also described withrespect to the description of other figures, including FIG. 14.

As shown in FIG. 1, the automated data consolidation module 100 may beattached to and portions can be stored within a module 101. The module101 can conveniently provide direct access to the patient 102. An IVpole 105 may provide a convenient mounting support location for theautomated data consolidation module for IV medications 100 (hereinafter,“automated data consolidation module 100”). In some examples, thecomponents and systems of the automated data consolidation module100 ofthis disclosure can be supported by other mounting supports, includingbut not limited to a boom-mounted rack system, a wheeled rack system anda bed 103 mounting bracket. One or more computers including processingcircuitry 157, of the automated data consolidation modu1e100 of thisdisclosure may be conveniently and safely housed inside the module 101.

In some examples, it is anticipated that some or all of the componentsof the automated data consolidation module100 of this disclosure couldbe used in other healthcare settings such as the intensive care unit,the emergency room or on the ward. As shown in FIG. 1, the module 101may be mounted on an IV pole 105 or other suitable mounting structurelocated near the patient 102.

In some examples, a touch-screen electronic record display 126 canconvert to a qwerty-type keyboard to allow uncommon anesthetic andsurgical events or deviations from pre-recorded scripts, to be manuallydocumented. In some examples, natural language speech recognitionsoftware may be included in the processing circuitry 157 allowing theoperator to simply dictate the record. This allows the standard computerkeyboard that is used for data entry in most electronic anestheticrecords, to be eliminated. Standard keyboards are known to becontaminated with pathogenic organisms and are nearly impossible toclean and decontaminate due to their irregular surfaces. In contrast,the smooth glass or plastic face of a touch-screen monitor is easy toclean with no crevasses to hide organisms.

In some examples, the automated data consolidation module for IVmedications 100 of this disclosure can include a system forautomatically measuring and recording the administration of IVmedications. In some examples, the system for automatically measuringand recording the administration of IV medications includes a medicationidentification and measurement system 128. In some examples, aspects ofthe automated data consolidation module100 can be provided together withor separately from other aspects of the IV medication identification andmeasurement system 128 (hereinafter, “medication identification andmeasurement system 128”). Likewise, aspects of the medicationidentification and measurement system 128 can be provided together withor separately from other aspects of the automated data consolidationmodule 100.

FIG. 2 illustrates an isometric view of an example automated dataconsolidation module 200 for generating an automated electronicanesthetic record located proximate to a patient 202. Features of theautomated data consolidation module 100 of FIG. 1 may be included in theautomated data consolidation module 200 of FIG. 2, and vice-versa,therefore all aspects may not be described in further detail. Likenumerals can represent like elements. Aspects of FIGS. 1 and 2 may alsobe described together. Some aspects of FIG. 2, including an IV fluididentification and measurement system 130, are described with respectother figures, including the description of IV fluid identification andmeasurement system 1430 of FIG. 14.

As shown in FIG. 2, an example medication identification and measurementsystem 228 may be attached to a relocation module 201 that may beadvantageously positioned proximate the patient 202, such as near thepatient's head on a surgical table 212. In this position medications canbe conveniently administered by medical personnel while also tending toand observing the patient 202 during surgery.

It would be difficult or even impossible to manage the uncontained wasteheat produced by electronic and electromechanical equipment mounted on asimple open rack because it can escape in any direction. In someexamples, the automated data consolidation module100,200 can include acowling (e.g., 299C; FIG. 2) covering substantially the entire outersurface of the housing 299. The cowling 299C not only protects theequipment from accidental fluid damage but also confines the waste heatfrom the electronic and electromechanical equipment mounted within theautomated data consolidation module 100,200 to the inside of the module201 and cowling 299C. In some examples, the confined waste heat can thenbe safely managed.

In some examples, the cowling 299C cover of the automated dataconsolidation module 100,200 can form or support a waste heat managementsystem. In some examples, the cowling 299C can be provided on an innersurface of the housing 299. In some examples, the cowling 299C can bedescribed as an insulation. In some examples, the housing 299 caninclude other types of insulation from heat and/or water. Any suitabletype of insulated housing suitable for use in a surgical field can beprovided.

In some examples, the medication identification and measurement system128 (FIG. 1), 228 (FIG. 2) may include one or more sensors, such as oneor more of: a barcode reader or QR code reader (e.g., 436, FIG. 4), aradio-frequency identification (RFID) interrogator (e.g., 438, FIG. 4),or any other suitable sensor for accurately and reliably identifying amedication for IV administration. As defined herein, a barcode readercan include any other type of identifying reader, such as, but notlimited to, a QR code reader. Likewise, the RFID interrogator can be anytype of interrogator and is not limited to those interrogators based onradio frequency. Examples of such sensors are described herein, such asin FIGS. 4 and 5.

In some examples, instead of, or in addition to one or more of an RFIDinterrogator 438 and a barcode reader 436, the medication identificationand measurement system 128, 228 can receive an input to determine theidentity. For example, the medication identification and measurementsystem 128, 228 can include one or more of: a sensor, such as barcodereader 436 of FIG. 4, configured to identify the one or more IVmedications or fluids, or an input configured to receive the identity ofthe one or more IV medications or fluids, such as via the anestheticrecord input component 224.

In some examples, the barcode reader (e.g., 436, FIG. 4) may be a“computer vision” or “machine vision” camera with the capability ofreading barcodes. The term “machine vision” is often associated withindustrial applications of a computer's ability to see, while the term“computer vision” is often used to describe any type of technology inwhich a computer is tasked with digitizing an image, processing the datait contains and taking some kind of action. In this disclosure the terms“machine vision” and “computer vision” may be used interchangeably.Traditionally, machine vision includes technology and methods used toprovide imaging-based automatic inspection and analysis, processcontrol, and robot guidance. Machine vision is sometimes used inmanufacturing environments. Machine vision refers to many technologies,software and hardware products including processing circuitry,integrated systems and methods.

The inventors have discovered that machine vision can be useful beyondits traditional uses. The inventors discovered that machine vision canbe advantageous in implementing an automated data consolidation module100, 200 because it offers reliable measurements, gauging, objectrecognition, pattern recognition and liquid fill level measurements.Machine vision does not get tired or distracted. Machine vision excelsat quantitative measurement of a structured scene because of its speed,accuracy and repeatability. However, it may require the scene to bestructured to perform the desired function.

Machine vision can be very accurate for measuring size of an object at aknown distance or the distance of an object of known size. However, itcannot do both. Therefore, in some examples it is important to know theexact location of a syringe (e.g., 406, FIG. 4) and thus know thedistance from the camera (e.g., 436, FIG. 4) to the syringe (e.g., 406,FIG. 4) in order for the machine vision to calculate the distance of themovement of the plunger (e.g., 446, FIG. 5) within the syringe (e.g.,406, FIG. 5). This is what we mean by the “scene being structured.”

Machine vision may be advantageous for the automated data consolidationmodule 100, 200 of this disclosure because it “sees” and measures, butdoes not touch or interfere with the healthcare provider doing theirnormal job of injecting medications or administering IV fluids. Further,the same visual image that is used by the machine vision software can betransmitted and displayed on a screen 126, 226 to give the operator(whose fingers can be pushing the plunger 446 of the syringe 406, aclose-up view of the syringe 406. FIG. 5 is a cross-section view takenat 5-5 of FIG. 4. The machine vision camera 436 can be looking at thesame view of the syringe 406 as the operator and it is the same orsimilar view that the operator would see if they were injecting IVmedications the traditional way.

The machine vision camera, or digital camera, can include machine visionsoftware, or the machine vision camera can be in electricalcommunication with (e.g., operably coupled to) one or more hardwareprocessors, such as processing circuitry 157, 257 and one or moremachine-readable mediums 159, 259. The one or more machine-readablemediums 159, 259 can include instructions (e.g., software), that whenimplemented on the processing circuitry 157, 257, can perform thefunctions described herein. The processing circuitry 157, 257 can bestored in the module 101, relocation module 201 or remote from themodules 101, 201 (e.g. in a wired or wireless manner). The one or moremachine-readable mediums 159 can be a storage device, such as a memorylocated in the module 201 or remote from the module 101, 201.

In some examples, the RFID interrogator 438 may be either High Frequency(HF) or Near Field (NF) RFID in order to advantageously limit theread-range to a distance of less than 12 inches. In some examples, theRFID read-range may advantageously be limited, such as to less than 8inches so that only a specific medication injection is identified at anytime. In a possibly more preferred example, the RFID read-range may belimited to less than 4 inches to further prevent mis-readings. NF-RFIDhas a short read-range by definition and the read-range of HF-RFID canbe easily limited by restricting the size of the antenna on the tag. Incontrast, longer read-range RFID such as Ultra-high Frequency (UHF-RFID)may confusingly interrogate every RFID tag in the operating room andthus be unable to identify which medication is being delivered to themedication identification and measurement system 128, 228. However, anysuitable RFID range for a particular application may be used.

FIG. 3 illustrates a plan view of an example of preloaded syringes 306Aand 306B that can be used with the automated data consolidation module200 of FIG. 2.

The one or more preloaded syringes 306A and 306B may be labeled with aunique barcode label 307 or an RFID tag 308 that may identify one ormore of the drug, the concentration, the lot number, the expirationdate, the manufacturer and other important information. In someexamples, a unique barcode label 307 may be a “2-D” barcode label inorder to include more information on a smaller area than traditionalbarcode labels. In some examples, the barcode label 307 or RFID tag 308includes the drug identifying label 309A and 309B for convenient use bythe caregiver.

In some examples, the syringes 306A and 306B can be filled at the pointof use and may be labeled with drug labels 309A and 309B and eitherbarcode labels 307 or RFID tags 308 that are removably attached to thedrug bottle or vial at the factory or pharmacy. The drug labels 309A and309B and either barcode labels 307 or RFID tags 308 may be easilyremoved from the drug bottle or vial and adhesively attached to thesyringe 306A or 306B at the time that the syringe 306A or 306B is loadedwith the drug by the caregiver. Instead of, or in addition to thebarcode labels 307 or RFID tags 308, any other suitable “tag/reader”system known in the arts, may be used.

FIGS. 4-10 illustrate examples of medication identification andmeasurement systems 428, 628, 828 that can be used with the automateddata consolidation module 100, 200 of FIGS. 1 and 2. However, aspects ofthe medication identification and measurement systems 428, 628 and 828may be used with other systems, and other medication identification andmeasurement systems may be used with the automated data consolidationmodule 100, 200. Furthermore, some examples of the automated dataconsolidation module 100, 200 can omit aspects of the medicationidentification and measurement systems, or can omit a medicationidentification and measurement system altogether. FIG. 4 illustrates aportion of an automated data consolidation module 400 including a sideview of an example medication identification and measurement system 428and a syringe 406 that can be used with the automated data consolidationmodule 100, 200 of FIGS. 1 and 2, to monitor drug delivery. FIG. 5illustrates a cross-sectional view of the medication identification andmeasurement system 428 and the syringe 406 (not shown in cross-section)of FIG. 4, taken along line 5-5. FIGS. 4 and 5 are described together.

As shown in FIGS. 4 and 5, the medication identification and measurementsystem 428 may include at least one injection portal 411. The injectionportal 411 may be a receptacle for accommodating a syringe 406 in afixed and known location and can be configured to orient the Luer taperconnector 513 to mate with an injection port 515. The injection port 515can be secured within the injection portal 411 and can be in fluidcommunication with IV tubing 520. In some examples, the injection portal411 may include an injection portal tube 416, such as a transparent tubethat is sized to receive and accommodate a syringe barrel 418 of asyringe 406. In some examples, the injection portal can be configured toreceive a specific size syringe barrel 418. In some examples, multipleinjection portals 411 can be provided to accommodate syringes 406 ofdifferent sizes.

FIG. 6 illustrates a portion of an automated data consolidation module600 including a side view of a second example of a medicationidentification and measurement system 628 and a syringe 606 that can beused with the automated data consolidation module 100, 200 of FIGS. 1and 2, to monitor drug delivery. FIG. 7 illustrates a cross-sectionalview of the second example of a medication identification andmeasurement system 628 and the syringe 606 (not shown in cross-section)of FIG. 6, taken along line 7-7. FIGS. 6 and 7 are described together.

As shown in FIGS. 6 and 7, the injection portal 611 of the medicationidentification and measurement system 628 may be large enough toaccommodate syringes 606 of multiple sizes within the space defined by areal or imaginary injection portal tube 616. In this example, accuratelyorienting the Luer taper connector 713 to mate with an injection port715 may be accomplished by one or more orienting members such as one ormore spring positioning members 622A-F that engage with the syringebarrel 618 to center it in the injection portal 611. In some examples,there may be two or more rows of spring positioning members 622A-F. Forexample, spring positioning members 622A, B, E, F may be located nearthe entrance to the injection portal 611 and spring positioning members622C, D may be located near the injection port 715 to assure accuratepositioning for mating with the Luer taper connector 713. Springpositioning members 622A-F may include not only spring wires or metal orpolymer or plastic spring pieces but any flexible material orcombination of materials or shapes that can be deformed by the syringebarrel 618 entering the injection portal 611 and retain a memory (e.g.,elastically deformable, substantially elastically deformable,resiliently deformable, resilient member) so as to urge the syringebarrel 618 into a centered position within the space defined by a realor imaginary injection portal tube 616.

One objective of the spring positioning members 622A-F can be to“automatically” center and align the Luer taper connector 713 of thesyringe 606 with the injection port 715, so that the operator can simplyand conveniently push the syringe 606 into the injection portal 611 andno further manual alignment may be needed. The spring positioningmembers 622A-F can also obviate the need for the operator to toucheither the Luer taper connector 713 of the syringe 606 or the injectionport 715, thus beneficially preventing accidental infectiouscontamination by the operators' fingers and gloves.

FIG. 8 illustrates a portion of an automated data consolidation module800 including a side view of a third example of a medicationidentification and measurement system 828 and a syringe 806 that can beused with the automated data consolidation module 100, 200 of FIGS. 1and 2. FIG. 9 illustrates a cross-sectional view of the third example ofa medication identification and measurement system 828 and the syringe806 (not shown in cross-section) of FIG. 8, taken along line 9-9. FIGS.8 and 9 are described together.

As shown in FIGS. 8 and 9, a syringe barrel 818 may be centered and heldin place by one or more orienting members, such as compressionpositioning members 842A,B. The compression positioning members 842A, Bmay be urged apart by inserting the syringe barrel 818 there between.Springs 844A-D can compress and create a pressure pushing thecompression positioning members 842A,B against syringe barrel 818. Thecompression positioning members 842A,B shown in FIGS. 8 and 9 are merelyillustrative, and many other sizes, shapes, numbers and locations ofcompression positioning members 842 are anticipated.

Compression positioning members 842A,B may be simple spring 844A-Dactivated devices (e.g., resilient members) as shown in FIGS. 8 and 9 ormay be any mechanism that can expand (e.g., resiliently expand) toaccommodate syringe barrels of various sizes and urge the syringe barrel818 into a centered position within the space defined by a real orimaginary injection portal tube 816. This example shows spring 844A-Dactivated compression positioning members 842A,B but many othermechanical activation mechanisms are anticipated. The compressionpositioning members 842A,B can be elastically deformable, substantiallyelastically deformable, resiliently deformable, include one or moreresilient members.

Other examples of positioning members designed to hold an insertedsyringe 806 in the center of the injection portal 811 and thus orientingthe Luer taper 913 for mating with the injection port 915 areanticipated. Positioning the inserted syringe 806 in the center of theinjection portal 811 allows the machine vision to work from a knowndistance and thus calculations of syringe plunger 948 movement can bevery accurate.

In some examples, instead of the positioning members shown in theexamples of FIGS. 6-9 holding a syringe centrally, the positioningmembers 622A-F or 842A,B can be designed to hold an inserted syringe606, 806 at a known, but off center position in the injection portal611, 811, such as when the injection port 715, 915 (FIGS. 7 and 9) ispositioned off center in the injection portal 611, 811. Any arrangementof at least one positioning member that aligns an inserted syringe at aknown position may be provided.

In some examples, and as shown in FIGS. 4, 6 and 8 the medicationidentification and measurement system 428, 628, 828 of this disclosuremay include one or more “machine vision” cameras 436, 636, 836 thatinput digital images into one or more processors having processingcircuitry 157, 257 as shown and described in FIGS. 1, 2, that isprogramed to analyze machine vision images. In some examples, one of theimages tha the machine vision cameras 436, 636, 836 may “see” is abarcode label 307 on the syringe 406, 606, 806, that has been insertedinto the injection portal 411, 611, 811, for identifying the medicationin the syringe 406, 606, 806. As previously noted, the barcode label 307can identify the brand name and/or generic name of the medication in thesyringe. In some examples, the barcode label 307 also may identify oneor more of the concentration of the medication, the lot number, theexpiration date and other information that may be useful for inventorymanagement.

As shown in FIGS. 4, 6 and 8, the automated data consolidation module400, 600, 800 of this disclosure can include one or more radio frequencyidentification (RFID) interrogation antennas 438, 638, 838 that inputRFID information into a processor, such as processing circuitry 157, 257as shown and described in FIGS. 1 and 2, that is programed to analyzeRFID data. In some examples, the RFID interrogation antennas 438, 638,838 can interrogate a RFID tag 308 (FIG. 3) attached to the syringe 406,606, 806, that has been inserted into the injection portal 411, 611,811, for identifying the medication in the syringe 406, 606, 806. Insome examples, short range RFID such as near field (NF) or highfrequency (HF) may be advantageous because they may only detect thesyringe 406, 606, 806 that is adjacent to or inside the security systemfor IV medications 400, 600, 800, and not detect the various othermedication syringes that may be sitting on the worktable such as206A-206C in FIG. 2.

As shown in FIGS. 4, 6 and 8 the medication identification andmeasurement system 428, 628, 828 of this disclosure may include a RFIDinterrogator 438, 638, 838. In some examples, the RFID interrogator 438,638, 838 that can include antennas that may be located inside themedication identification and measurement system 428, 628, 828 . In someexamples, the RFID interrogator antennas 438, 638, 838 may be locatedexternal to but proximate the medication identification and measurementsystem 428, 628, 828. As the syringe 406, 606, 806 is brought intoproximity of the medication identification and measurement system, theRFID interrogator 438, 638, 838 can interrogate the RFID tag 308 on thesyringe 406, 606, 806, thereby accurately and reliably identifying amedication for IV administration. In some examples, the RFID tag 308 orother marker may include one or more of: the generic and brand name ofthe drug, the concentration, the lot number, the expiration date, themanufacturer and other important information that may be recorded. Insome examples, the generic and brand name of the drug and theconcentration of the drug can be displayed in the injection section of adisplay such as the display 126, 226 (FIGS. 1, 2).

Machine vision is very accurate for measuring the size of an object at aknown distance or the distance of an object of known size. However, itcannot do both. Therefore, in some examples it is important to know theexact location of a syringe and thus know the distance from the camerato the syringe in order to accurately calculate the distance of themovement of the plunger within the syringe.

Syringes are available in multiple sizes such as 3 cc, 6 cc and 12 cc,each of which is a different diameter. The machine vision processor mustknow both the internal diameter of the barrel of the syringe and thedistance that the syringe plunger moves down the barrel, in order tocalculate the volume of medication injected, unless it has anothersource of information. The machine vision of this disclosure can measurethe diameter of the syringe because in the examples the syringe 406,606, 806 is held at known distance and in a centered location relativeto the machine vision cameras 436, 636, 836. Alternately, the automateddata consolidation module 400, 600, 800 of this disclosure may beprogramed to know that the particular hospital uses only Monoject®syringes for example and the internal diameter of each Monoject® syringesize may be pre-programed into the computer. In this case, the machinevision only needs to differentiate 3 cc, 6 cc and 12 cc syringe sizesfrom each other. The machine vision processor can determine the internaldiameter of the barrel of the syringe. In some examples, the syringesize may be included in the information provided by the barcode 307 orRFID 308 (FIG. 3).

In some examples, such as the examples of FIGS. 4-9, the machine visionsystem, including the machine vision camera 436, 636, 836 and theprocessor 157, 257 of FIGS. 1 and 2 (e.g., processing circuitry) inelectrical communication with the machine vision camera 436, 636, 836,can visually detect and determine other geometry information about thesyringe 406, 606, 806 besides the outside diameter, such as determiningthe inside diameter, or the inner or outer length of the syringe. Themedication identification and measurement systems 428, 628, 828 can usethe geometry information to determine the size or type of the syringe406, 606, 806, or can use the geometry information to calculate a volumeof the syringe 406, 606, 806.

In some examples, as the syringe 406, 606, 806 is advanced into theinjection portal 411, 611, 811, the image of the syringe 406, 606, 806entering the injection portal 411, 611, 811 is displayed in real time inan injection section 126 a , 226 a of the display 126, 226 (FIGS. 1 and2). Therefore, the caregiver can watch the syringe 406, 606, 806 advanceand engage with the injection port 515, 715, 915. In some examples, theinjection portal tube 416, 616, 816 or the spring positioning members622A-E or the compression positioning members 842A,B, urge the syringe606, 806 into position to mate with the injection port 715, 915 but theactual connection can also be observed as it is happening by thecaregiver on the display 126, 226. Even though the caregiver is notphysically holding the injection port 515, 715, 915 as they typicallywould, they can watch the engagement of the Luer connector 513, 713, 913with the injection port 515, 715, 915 on a display 126, 226, the view isessentially identical to the thousands of injections that they have madeduring their career. In some examples, the actual image of the syringe406, 606, 806 can be displayed on the display 126, 226, while in otherexamples the data obtained by the camera 436, 636, 836 can be convertedto a representation of the syringe displayed on the display 126, 226.

In some examples, once the syringe 406, 606, 806 is securely connectedto the injection port 515, 715, 915, the caregiver pushes on the plunger446, 646, 846 of the syringe 406, 606, 806, injecting the medicationinto the injection port 515, 715, 915 and IV tubing 520, 720, 920. Thecaregiver can visualize the plunger seal 548, 748, 948 move down thesyringe barrel 418, 618, 818 and can determine the volume of medicationinjected by the graduated markings on the syringe 406, 606, 806. Thus,the engagement of the Luer connector 513, 713, 913 with the injectionport 515, 715, 915 and the injected volume are observed by the caregiveron the display 126, 226 and the traditional method and routine ofinjection is minimally altered by implementing the automated dataconsolidation module 100, 200 including the example medicationidentification and measurement systems 428, 628, 828.

In some examples, the processing circuitry 157, 257 (FIGS. 1 and 2) or acomputer may also simultaneously generate data representing a runningtotal of the volume and dosage of the injected medication and cantransmit the generated data to the display 126, 226 to display volumeand dosage information on the display 126, 226. In some examples, theprocessing circuitry 157, 257 or a computer may also generate its owngraduated scale and transmit the generated graduated scale informationto the display 126, 226 to superimpose the scale on the image of thesyringe 406, 606, 806 or next to the image of the syringe 406, 606, 806,for added visual clarity of the injected volume and dose.

In some examples, the machine vision determination of the injectedvolume may be calculated by multiplying the internal cross-sectionalarea of the syringe (πr²) by the distance that the syringe plungermoves. The radius of the syringe may be determined in one or more ways.For example, the machine vision function may determine that the syringeapproximates a 3 cc or 12 cc syringe and the computer is programed toknow that the hospital uses a specific brand of syringes and theinternal diameter (radius) of each of these syringe sizes is preciselyknown. (An example of diameter d, radius r is shown in FIG. 5) Anotherexample may require the machine vision camera to measure the outerdiameter of the syringe and then subtract an approximated wall thickness(either measured or known value stored in a memory) from the measureddiameter to determine the internal diameter. In another example, theinternal diameter of the syringe may be supplied to the processingcircuitry 157, 257 or a computer as part of the RFID 308 or barcode 307information. In another example, the machine vision may determine theinner diameter of the syringe by determining an outer diameter of theplunger as viewed through the transparent or semi-transparent syringeand determine the wall thickness, In yet another example, the machinevision may be able to visibly determine the inner diameter or radiusdirectly through the transparent or semi-transparent syringe. Any othersuitable determination, calculation or algorithm may be used todetermine the radius, diameter and injected volume.

In some examples, the machine vision determination of the distance thatthe syringe plunger 446, 646, 846 moves may be by “observing” themovement of the black rubber plunger seal 548, 748, 948 against thevisible scale printed on the syringe 406, 606, 806. In this example, themachine vision can be programed to recognize the markings on the syringe406, 606, 806.

In some examples, the machine vision determination of the distance thatthe syringe plunger 446, 646, 846 moves may be by observing the movementof the black rubber plunger seal 548, 748, 948 relative to a scalecalculated by the processing circuitry 157, 257 (FIGS. 1 and 2). Thegeometrical calculation of the scale that determines the distance thatthe syringe plunger 446, 646, 846 moves may be easiest to determinealong the widest part of the syringe that corresponds with the center C(FIG. 5) of the syringe 406, 606, 806, which is a known distance fromthe machine vision camera 436, 636, 836. Alternatively, thecomputer-constructed scale may be applied to the side of the syringe406, 606, 806 facing the camera 436, 636, 836, if the radius of thesyringe 406, 606, 806 is subtracted from the known distance to thecenter C (FIG. 5) of the syringe 406, 606, 806 in order to calculate thedistance 437 from the machine vision camera 436, 636, 836 to the nearside (e.g., 411A) of the syringe 406, 606, 806.

In some examples, the movement of the black rubber plunger seal 548,748, 948 of the syringe 406, 606, 806 can be clearly identifiable by themachine vision camera 436, 636, 836 and a scale to determine thedistance moved by the plunger 446, 646, 846 can either be “visualized”or constructed by the machine vision computer (e.g., processingcircuitry). Multiplying the distance that the plunger seal 548, 748, 948moves by the known or measured internal diameter d (FIG. 5) of thesyringe 406, 606, 806 and thus cross-sectional area of the plunger seal548, 748, 948, allows the processing circuitry 157, 257 or a computer inelectrical communication with the processing circuitry 157, 257 tocalculate an accurate injected volume. The measured injection volume anddosage may be displayed on the display 126, 226 of the module 101, 201(FIGS. 1 and 2). Without interfering with or changing theanesthesiologists' normal or traditional medication injection routines,an unobtrusive machine vision camera 436, 636, 836 and computer (e.g.,processing circuitry) can “observe” the medication injections andautomatically record them in the EMR.

In some examples, the injected volume of medication may be determined byother sensors or methods. For example, the systems described herein canemploy (e.g., substitute) other sensors such as a non-visual opticalsensor 436A in place of or in addition to the machine vision camera 436,636, 638 described in FIGS. 4-9. For example, a light source can shineon one or more light sensitive elements such as photodiodes, and theposition of the plunger of the syringe can be roughly determined by theobstruction of the light beam by the plunger. Other fluid measurementmethods can have a sensor including adding magnetic material to thesyringe plunger and detecting movement of the plunger with a magneticproximity sensor. Alternatively, fluid flow may be measured with fluidflow meters in the IV fluid stream. These examples are not meant to bean exhaustive list but rather to illustrate that there are alternativetechnologies to machine vision (e.g., sensors), for noncontactmeasurement (e.g., sensing) of fluid flow from a syringe that areanticipated in this disclosure.

In some examples, the injected volume of medication may be determined byother sensors or methods. For example, the systems described herein canemploy (e.g., substitute) other sensors in place of or in addition tothe machine vision cameras for either or both the medication and fluidflows. Other fluid flow sensors anticipated by this disclosure includebut are not limited to: ultrasonic flow meters, propeller flow meters,magnetic flow meters, turbine flow meters, differential pressure flowmeters, piston flow meters, helical flow meters, vortex flow meters,vane flow meters, paddle wheel flow meters, thermal flow meters,semicylindrical capacitive sensor flow meters and Coriolis flow meters.

Securing the injection port 515, 715, 915 within the injection portal411, 611, 811 prevents the caregiver from touching the injection port515, 715, 915. Normally caregivers wear gloves to protect themselvesfrom infectious contaminates from the patient and operating room andtheir gloves are nearly always contaminated. Anything they touch will becontaminated. They typically pick up and hold the IV injection port 515,715, 915 with one hand while inserting the Luer taper connector 513,713, 913 of the syringe 406, 606, 806 into the injection port 515, 715,915. In the process, the injection port 515, 715, 915 is frequentlycontaminated with pathogenic organisms from their gloves that can enterthe patient's blood stream with the next injection, causing seriousinfections. It is therefore advantageous from the infection preventionpoint of view, if the Luer connection and injection can be accomplishedwhile never touching the injection port 515, 715, 915.

In some examples as shown in FIGS. 1 and 2, the automated dataconsolidation module 100, 200 may include an external reader, such asbarcode reader 180, 280 on the module 101, 201 to read a barcode, QRcode or the like for identification. This barcode reader 180, 280 may beused to identify the healthcare provider injecting a medication byreading a barcode or QR code 1186 on the user's ID badge for example(FIG. 11). In some examples as shown in FIGS. 1 and 2, the automateddata consolidation module 100, 200 may include an external RFID reader182, 282 on the module 101, 201. This RFID reader 182, 282 may be usedto identify the healthcare provider injecting a medication by reading anRFID tag 1188 on the user's ID badge 1184B for example (FIG. 11). Insome examples as shown in FIGS. 4, 6 and 8, the automated dataconsolidation module 400, 600, 800 may include an internal RFID reader438, 638, 838 in the module 101, 201. This RFID reader 438, 638, 838 mayalso be used to identify the healthcare provider injecting a medicationby reading an RFID tag on the user's ID badge for example.

It is an important part of the record to know who injected themedication and their identity can be easily verified and documented bythe automated data consolidation module 400, 600, 800 using eitherbarcode, QR code or RFID. Other identification technologies are alsoanticipated.

FIG. 10 illustrates an example injection port cassette 1054 that can beused with the automated data consolidation module 100, 200 of FIGS. 1and 2, as detailed in FIGS. 5, 7 and 9. As shown in FIG. 10, theinjection port 1015 may be mounted on an injection port cassette 1054 inorder to make the attachment to the automated data consolidation module100, 200, 400, 600, 800 easier and more secure. The injection portcassette 1054 may be a piece of molded polymer or plastic onto which theinjection port 1015 and IV tubing 1020 may be attached. The injectionport cassette 1054 may be shaped and sized to fit into a slot in theautomated data consolidation module 100, 200, 400, 600, 800. When theinjection port cassette 1054 is fit into a slot in the automated dataconsolidation module 100, 200, 400, 600, 800, the injection port 1015can be positioned substantially in the center of the injection portal411, 611, 811 for mating with the Luer tapers 513, 713, 913. Theinjection port cassette 1054 can also be configured to be removed intactfrom the automated data consolidation module 400, 600, 800 so that thepatient can be transferred and the IV tubing 1020 can be moved with thepatient and continue to operate normally.

In some examples, and as shown in FIG. 10, the injection port cassette1054 can include an IV bypass channel 1056 in the IV tubing 1020. The IVbypass channel 1056 can allow the IV fluids to flow unencumbered by themedication injection apparatus. The injection port cassette 1054 caninclude a medication channel 1058 in the IV tubing 1020 and, themedication channel 1058 may include one or more stop-flow clamps1060A,B. The one or more stop-flow clamps 1060A,B may be activated bythe automated data consolidation module100, 200, 400, 600, 800 if amedication error is identified. The one or more stop-flow clamps 1060A,Bmay be powered by one or more electromechanical solenoids that squeezethe IV tubing in the medication channel 1058 flat, obstructing the flow.Other electromechanical flow obstructers are anticipated.

In some situations, such as when administering a drug to a patientallergic to that drug, or administering potent cardiovascular drugs to apatient with normal vital signs, or administering a drug with a likelymistaken identity, the computer, such as processing circuitry 157, 257(FIGS. 1 and 2) for the automated data consolidation module 100, 200,400, 600, 800 of this disclosure can automatically activate thestop-flow clamps 1060A,B to compress the medication channel 1058 tubingupstream and/or downstream from the injection port 1015. Compressing theIV tubing both upstream and downstream from the injection port 1015prevents the injection of any medication into the IV tubing 1020. Analert to the adverse condition of the injection may be displayed ondisplay 126, 226 where the stop-flow condition can be over-ridden by theoperator touching a manual override switch on the display 126, 226 orthe module 101, 201, if the injection was not erroneous. While thestop-flow can occur in the medication channel 1058, the IV fluid flowcan continue normally in the parallel bypass channel 1056.

The stop-flow clamps 1060A,B can allow the processing circuitry 157, 257(e.g., processor, hardware processing circuitry) of the automated dataconsolidation module 100, 200, 400, 600, 800 to not only warn theoperator of a pending medication error, but physically prevent theinjection. Perhaps equally as important is that the stop-flow clamps1060A,B can be quickly released by the operator touching a manualoverride switch in the event that the apparent error was in fact aplanned event or otherwise desired by the operator.

In some examples, a part of the automated data consolidation module 100,200, 400, 600, 800 can include the ability for the computer (e.g.,machine) to know the patient's medical history, medication orders, vitalsigns, current medications, medication orders and other importantinformation about that patient. In some examples, the medication insyringe 306 can be identified by RFID tag 308 (FIG. 3) and is detectedby RFID interrogator 438, 638, 838 as the syringe 306 enters injectionportal 411, 611, 811. The processing circuitry (e.g., 157, 257) of theautomated data consolidation module 100, 200, 400, 600, 800 cancross-reference the proposed injection to the patient's medical history,medication orders, vital signs, current medications and other importantinformation about that patient, providing a safety “over-watch” guardingagainst medication errors. In some examples, the processing circuitry(e.g., 157, 257) of the automated data consolidation module 100, 200,400, 600, 800 may include algorithms and/or “artificial intelligence”that can provide alternative medication suggestions based on patient'smedical history, medication orders, vital signs, current medications andother important information about that patient.

In some examples, the EMRs that were created by the automated dataconsolidation module 100, 200, 400, 600, 800 of this disclosure canprovide accurate and temporally correlated information about therelationship between any injected medication and the resultingphysiologic response. This is uniquely accurate dose-response data. Insome examples, the EMRs that were created by the automated dataconsolidation module 100, 200, 400, 600, 800 for hundreds of thousandsor even millions of patients, may be aggregated and analyzed as “bigdata.” The “big data” from these EMRs may be used for a variety ofpurposes including but not limited to medical research, patient andhospital management and the development of “artificial intelligence”algorithms that can provide alternative medication suggestions. Ongoing“big data” from more and more EMRs can be used to continually improveand refine the “artificial intelligence” algorithms, much like the“artificial intelligence” algorithm development process being used todevelop self-driving vehicles. These “artificial intelligence”algorithms can be used to provide automated (“self-driving” or“partially self-driving”) anesthesia during surgery or automatedmedication delivery.

FIG. 11 illustrates a plan view of an example of healthcare provider IDbadges 1184A and 1184B that can be used with the system of FIGS. 1 and2, in accordance with at least one example. In some examples, the “chainof custody” may begin by electronically identifying the healthcareprovider as the drugs are checked out from the pharmacy or Pixismedication dispenser. In some examples and as shown in FIG. 11, eachprovider can have a personalized RFID tag 1186, attached to theirhospital ID badge 1184A. In some examples each provider can have apersonalized barcode 1188, attached to their hospital ID badge 1184B forexample. When the drugs are checked out, the personalized RFID tag 1186may be read by an RFID interrogator or the personalized barcode 1188read by a barcode reader in the pharmacy and the ID of the providerchecking the drugs out may be noted in the hospital's computer and/orthe processing circuitry 157, 257 (FIGS. 1 and 2) for the automated dataconsolidation module 100, 200, 400, 600, 800 (FIGS. 1, 2, 4, 6 and 8).The specific RFID 308 or barcode 307 (FIG. 3) identification of theinjectable drug may also be recorded before the drug leaves the pharmacyand that information may be transmitted to the processing circuitry 157,257 of the automated data consolidation module 100, 200, 400, 600, 800of this disclosure. In some examples, instead of an RFID tag 1186 andRFID reader, other provider identification information and sensors foridentifying the provider can be used, such provider identificationinformation may include: a barcode, a QR code with the sensor being ableto read such codes. In other examples, the sensor can include a retinalscanner, fingerprint reader or a facial recognition scanner thatidentifies the provider by personably identifiable information (e.g.,provider identification information) may be used.

In some examples, when the provider arrives at the patient's bedside,the provider may be identified by their ID badge 1184 A,B. In someexamples, an ID badge 1184A that has an RFID tag 1186 may be read byRFID reader 182, 282 (FIGS. 1 and 2) that can be located on theautomated data consolidation module 100, 200, 400, 600, 800 or by RFIDreader 438, 638, 838 (FIGS. 4, 6 and 8) located inside the automateddata consolidation module 100, 200, 400, 600, 800 (FIGS. 1, 2, 4, 6 and8). In some examples, an ID badge 1184B that has a barcode 1188 may beread by barcode reader 180, 280 that can be located on the automateddata consolidation module 100, 200, 400, 600, 800. In some examples, aretinal scanner may be located on or near module 101, 201 in order topositively identify the provider by their retinal vasculature. Otherscanners including but not limited to facial recognition scanning arealso anticipated in order to positively identify the provider doing theinjection in order to automatically document this information to an EMRor other record.

In some examples, the provider's photograph may be taken by camera 190,290 (FIGS. 1 and 2) for further identification before allowing theinjection of scheduled drugs. In some examples, the camera 190, 290 maybe triggered, such as by processor 157, 257 (FIGS. 1 and 2), when asyringe e.g., 306B filled with a scheduled drug such as a narcotic isidentified as it enters the injection portal 411, 611, 811 and the RFIDinterrogator 438, 638, 838 interrogates the RFID tag 308 (FIG. 3) or themachine vision camera 436, 636, 836 reads the barcode 307 (FIG. 3) onthe syringe 306A. Non-scheduled medications may not need the addedsecurity of a photograph.

In some examples as shown in FIGS. 1 and 2, the automated dataconsolidation module 100, 200 of this disclosure may include a remotemonitor with display 187, 287. The remote monitor may include a wired orwireless connection to the automated data consolidation module 100, 200and may display some or all of the information shown on the electronicrecord display 126, 226, or other information generated by the automateddata consolidation module 100. For example, the processing circuitry157, 257 can be in electrical communication with the remote display 187,287 and the processing circuitry 157, 257 can send instructions to theremote display 187, 287 to display the generated information.

The remote monitor may be in the next room or miles away. The remotemonitor may allow remote supervision of healthcare delivery. Forexample, anesthesiologists frequently supervise up to four surgicalanesthetics at once, each being delivered by a nurse anesthetist. Inthis case, the anesthesiologist carrying a wireless remote monitor 187,287 can have real-time data on each case under their supervision.Similarly, a nurse anesthetist working in a rural hospital may besupervised by an anesthesiologist who is 50 miles away.

In some examples, the remote monitor with display 187, 287 can create arecord for billing. For example, when an anesthesiologist is supervisingmultiple anesthetics at once, the payers may dispute the involvement ineach case and refuse to pay. The remote monitor with display 187, 287may include an RFID reader that documents the close proximity of an RFIDtag on the anesthesiologist's ID badge. Any other type of proximitysensor may be used in place of the RFID tag, including but not limitedto GPS location sensing. Documenting that the anesthesiologist wascarrying the remote monitor with display 187, 287 throughout the time ofthe surgery is very good evidence that the anesthesiologist was activelyparticipating in the care of the patient.

In some examples, the remote monitor with display 187, 287 allowslong-distance medical consultation. For example, an expert at the MayoClinic could consult with a physician halfway around the world in Dubai,responding to real-time patient data displayed on the remote monitorwith display 187, 287.

FIG. 14 illustrates a side view of an example IV fluid identificationand measurement system 1430 that can be used with the systems of FIG.1-9, and the injection port cassette of FIG. 10. Some aspects of FIGS.1-14 are described together, however, the examples are merelyillustrative and the features can be used in any suitable combination.In some examples, the automated data consolidation module 100, 200 ofthis disclosure includes a system for automatically measuring andrecording the administration of IV fluids. To accomplish this, as shownin FIGS. 1, 2 and 14, the automated data consolidation module 100, 200can include an IV fluid identification and measurement system 130, 230,1430. In some examples, the IV fluid identification and measurementsystem 130, 230, 1430 can be mounted onto module 101.

Alternately, the IV fluid identification and measurement system 130,230, 1430 may be mounted to an IV pole 105 or racking system independentfrom the module 101. The system for automatically measuring andrecording the administration of IV fluids is not limited to use inanesthesia or in the operating room, but has applicability for usethroughout the hospital and other health care settings, including butnot limited to the ICU, ER, wards, rehabilitation centers and long termcare settings. In some examples, aspects of the IV fluid identificationmeasurement system 130, 230, 1430 can be provided alone or together withother features of the automated data consolidation module 100, 200including the medication identification and measurement system 128, 228.

In some examples, the IV fluid identification and measurement system130, 230, 1430 may be configured to accommodate one or more bags of IVfluid 132, 232A,B and 1432A,B. Each bag of IV fluid can include a dripchamber 134, 234A,B and 1434A,B and IV tubing 120, 220A,B and 1420A,B.IV flow rates may be controlled with the traditional manually operatedroller clamp that variably pinches the IV tubing 120, 220A,B and 1420A,Bto control or even stop the flow of IV fluids. In some examples, IV flowrates may be controlled with the automatically operatedelectromechanical flow rate clamps 1478 that variably pinch the IVtubing 1420A to control or even stop the flow of IV fluids. Theautomatically operated electromechanical flow rate clamps 1478 may becontrolled by the processing circuitry 157, 257, such as an electronicanesthetic record computer in module 101 or by any other suitableprocessor including hardware processing circuitry that is in electricalcommunication with the IV fluid identification and measurement system130, 230, 1430 and/or is located within the IV fluid identification andmeasurement system 130, 230, 1430.

In some examples, the IV fluid identification and measurement system130, 230, 1430 is configured to automatically measure and record theadministration of IV medications and fluids. The system 130, 230 caninclude one or more of a barcode reader and an RFID interrogator (suchas 1436A,B) for accurately and automatically identifying a fluid for IVadministration. Because of the close proximity to the adjacent bags,barcode identification may be preferable in order to prevent an RFIDinterrogator from reading the RFID tag on a neighboring bag. In someexamples, as shown in FIG. 14, one or more barcode labels 1405A,B may beapplied to the IV bags 1432A,B in a location where they can be read by asensor such as a barcode reader or machine vision camera 1436A,B, oranother machine vision camera located in a suitable position to read thebarcode. In some examples, a dedicated barcode reader or a machinevision camera may be positioned adjacent the barcode label 1405A,Blocation, specifically for reading the barcode label 1405A,B.

In some examples, the drip chamber 234A,B and 1434A,B of the IV set canbe positioned adjacent the one or more machine vision cameras 1436A,B.In some examples, a standard background 1468A,B may be positioned on theopposite side of the drip chamber 1434A,B from the machine visioncameras 1436A,B. The standard background 1468A,B may be a plainbackground or may be an advantageous color, pattern, color design orillumination that highlights each of the falling drops, for easieridentification by the processing circuitry 157, 257 (e.g., 1502, FIG.15). The machine vision software including instructions can be stored onone or more machine-readable mediums (such as 1522 in FIG. 15) that whenimplemented on hardware processing circuitry (including but not limitedto processing circuitry 157, 257) or in electrical communication withthe system, can perform the functions described herein. An example ofsuch electrical connection is shown by the connection of processor 1502with mass storage 1516 in FIG. 15.

In some examples, the processing circuitry 157, 257, 1502 can beconfigured to look for (e.g., sense, monitor, detect) a fluid meniscus1464A,B in the drip chamber 1434A,B. In this case “seeing” a fluidmeniscus 1464A,B indicates that there is fluid in the drip chamber1434A,B and therefore the IV bag 1432A,B is not empty, and air is notinadvertently entering the IV tubing 1420A, 1420B.

In some examples, if the IV fluid identification and measurement system130, 230, 1430 fails to “see” a fluid meniscus 1464A,B meaning that thedrip chamber 1434A,B is empty and thus the IV bag 1432A,B is empty,stop-flow clamps 1460A,B can be automatically activated. For example,processing circuitry 157, 257 can send an instruction to activate thestop-flow clamps 1460A,B to compress the IV tubing 1420A,B in order toprevent air from entering the IV tubing 1420A,B. In some examples, theempty IV bag 1432A,B condition detected by the processing circuitry 157,257 can cause an alert to be displayed to the caregiver on theanesthetic record display 226, such as by sending an instruction to thedisplay.

The combination of the machine vision camera 1436A,B in electricalcommunication with processing circuitry (e.g., 157, 257, FIGS. 1 and 2)that executes instructions stored on a machine readable medium can countthe number of drops of fluid per unit of time in a drip chamber 1434A,Bto calculate or to estimate the flow rate of an IV. The size of the dripchamber 1434A,B inlet orifice determines the volume of liquid in eachdrop. The inlet orifices of standard drip chambers are sized to createdrops sizes that result in 10, 12, 15, 20, 45 and 60 drops per ml. Givena particular drop volume (size), 10 drops per ml for example, the system130, 230, 1430 (e.g., via sensors, processing circuitry and machinereadable medium) can count the number of drops falling in a known periodof time and use that data to calculate or to estimate the flow rate. Ifthese estimates were attempted by a human, they may be less accurate athigher flow rates (higher drop counts) because the drops are so fast, itcan be difficult to count the drops. Eventually, at even higher flowrates the individual drops become a solid stream of fluid and the flowrate cannot be visually estimated.

In some examples, the IV fluid identification and measurement system130, 230, 1430 is configured to look for falling drops of fluid 1462A,Bwithin the drip chamber 1434A,B. When drops 1462A,B have beenidentified, the machine vision system (e.g., machine vision camera 1436Aor 1436B operably coupled to processing circuitry 157, 257) may firstmeasure the diameter of the drop 1462A,B to determine which of thestandard drop sizes or volumes it is counting. Most hospitalsstandardize on several infusion set sizes, 10, 20 and 60 drops per ccfor example. Therefore, when these limited choices of infusion setbrands and sizes have been programed into the computer, the machinevision system only needs to differentiate between these choices, whichis much easier than accurately measuring the diameter of the drops.Unlike the human eye, the machine vision can accurately count thefalling drops even at high flow rates to calculate an IV fluid flowrate.

In some examples, the machine vision system, including the machinevision cameras 1436A,B and instructions 1524 (e.g., software) stored ona machine readable medium 1522 and implemented by hardware processingcircuitry 157, 257 does not “see” falling drops 1462A,B. In thissituation, either the fluid is flowing in a steady stream that is notidentifiable or the fluid has stopped flowing. In some examples, thesetwo opposite conditions can be differentiated by inserting a floatingobject 1466A,B (hereinafter, “float”) into the drip chambers 1434A,B. Insome examples, the float1466A,B may be a ball-shaped float 1466A,B. Insome examples, the float may be patterned or multi-colored to moreeasily identify movement or spinning of the float. In some examples, ifthe machine vision system cannot identify falling drops 1462A,B, it thenlooks to the float 1466A,B for additional information. If the float1466A,B is not moving or spinning, the fluid flow has stopped. If thefloat 1466A,B is moving or spinning and drops cannot be identified, thefluid is flowing in a steady stream and the flow rate cannot be measuredby machine vision. In this situation, the system can determine fluidflow using an alternate method.

In some examples, the IV fluid identification and measurement system130, 230, 1430 may be configured to accommodate one or more bags of IVfluid 132, 232A,B, 1432A,B and each of these IV bags may be hanging froman electronic IV scale 1472A,B (e.g., a weight, a physicalcharacteristic sensor). The electronic IV scale 1472A can measure thecombined weight of the IV bag and fluid 1432A, the drip chamber 1434Aand the IV tubing 1420A. The electronic IV scale 1472B can measure theweight or combined weight of one or more of the IV bag and fluid 1432B,the drip chamber 1434B, the IV tubing 1420B and a pressure infuser 1474.In both of these examples, the electronic IV scale 1472A,B canaccurately measure the change in combined weight that occurs due to thedrainage of the IV fluid from the IV bag 1432A,B. The change in weightper unit time can be converted to flow rates by processing circuitry157, 257 in electrical communication with the electronic IV scale 1472A,1472B, for example, by the processing circuitry 157, 257, 1502 anddisplayed on the electronic record display 126, 226.

In some examples, the calculated flow rates for each IV bag 1432A,B mayalso be displayed on one or more digital flow-rate displays 1476A,Bmounted on the IV fluid identification and measurement system 1430. Thedigital flow-rate displays 1476A,B may be small LED or LCD displays thatconveniently tell the operator the flow rate while they are manuallyadjusting the flow rate near the IV bags 1432A,B and drip chambers1434A,B. The digital flow-rate displays 1476A,B are particularlyconvenient when the IV fluid identification and measurement system 130,1430 is a free standing entity mounted on an IV pole 105 for examplewhile being used on the ward or ICU.

In some examples, when the falling drops 1462A,B cannot be detected andyet the floats 1466A,B are moving or spinning, the fluid is determinedto be flowing in a steady stream and the flow rate cannot be measured bymachine vision. In this case the electronic record computer mayautomatically query the change in weight per unit time as measured bythe electronic IV scale 1472A,B to determine the IV flow rate. At highflow rates, the change in weight per unit time as measured by theelectronic IV scale 1472A, B will most likely be more accurate thancounting drops, in determining the IV flow rate.

The IV flow rate as determined by the change in weight per unit time canalso be compared to the IV flow rates determined by counting drops toverify the accuracy of each method. Without interfering with or changingthe healthcare providers normal or traditional IV routines, anunobtrusive machine vision camera and computer can “observe” the IV flowrates and automatically record them in the EMR.

The module or automated EMR system 101, 201 of this disclosure maycapture anesthetic event data but it must be noted that the sametechnologies described herein for capturing anesthetic event data can beused throughout the hospital or outpatient health care system to captureand record medication administration, IV fluid administration, vitalsigns and patient monitor inputs, provider events and other data.Non-operating room heath care locations are included within the scope ofthis disclosure. While this disclosure focuses on the totality offunctions offered by module 101, 201, each of the individual functionscan be offered independently of module 101, 201.

The use of the term Electronic Anesthetic Record (EAR) as defined hereincan include any memory such as an electronic surgical record (ESR), oran electronic medical record (EMR), or an electronic health record(EHR),and is not limited to anesthetic or surgical applications. Aspectsof the modules 101,201 described herein can also be employed inrecovery, hospital room and long-term care settings.

In an example, the module 201 of FIG. 2, can include a housing 299having a lower section 299A and a tower-like upper section 299B, whereinthe lower section 299A can be configured to house unrelated wasteheat-producing electronic and electromechanical surgical equipment, andwherein the tower-like upper section 299B can be located on top of thelower section 299A. The module 201 can also include a cowling 299Cexternal or internal to the housing 299 that substantially confineswaste heat generated by the unrelated waste heat-producing electronicand electromechanical surgical equipment. In addition, the module 201can include a system for monitoring the administration of one or more IVmedications and fluids 228, 230. As shown in the combination of FIGS. 2,4, 5 and 14, the system 228, 230 can include any of: a barcode 436reader or an RFID 438 interrogator configured to identify the one ormore IV medications or fluids; a machine vision digital camera 436 tocapture an image of one or more of a syringe 406 or a drip chamber1434A,B; processing circuitry 257 operably coupled to the barcode reader436 or the RFID interrogator 438 (or the machine vision digital camera436) to receive the identity of the one or more IV medications orfluids, the processing circuitry 257 operably coupled to the machinevision digital camera 436 to receive the captured image and determine avolume of medication administered from the syringe or fluid administeredfrom an IV bag based on the image; and a display 226 operably coupled tothe processing circuitry 257, the display 226 configured to receiveinstructions from the processing circuitry 257 to output the identityand determined volume of medication administered from the syringe orfluid administered from an IV bag.

As described herein, “dose/response” can be a useful medical process.Dose-response includes giving something to the patient (a medicine forexample) or doing something to the patient (mechanical ventilation orsurgery for example)—the “dose”, and then observing the patient's“response.”

Healthcare data in general is manually entered, not automated.Healthcare data in general is also not granular (e.g., beat-by-beat,second-by-second, continuous) but rather very intermittent (e.g., every15 minutes, every 4 hours, every day etc.). In general, most data in theEMR are simply data that was on the old paper record, manually enteredinto the EMR.

Manual data entry of dose events, whether they are reporting medicationinjections, fluid infusions or changes in the ventilator settings or anyother “dose events” are laborious, distracting and universally dislikedby healthcare providers. Perhaps more important is that the dose eventsare not accurately recorded. First, there are innocent errors inrecording. Second, there are omissions that may be innocent or for lackof time or due to provider sloppiness. Finally, there is no temporalcorrelation between a dose event and a related response event becausethe dose event is most likely manually recorded minutes to hours afterthe dose event occurred (if it is recorded at all).

At this date, the only data that is automatic and timely recorded is the“response” data provided by the physiologic monitors. Even the monitorresponse data is usually not recorded beat-by-beat but ratherintermittently recorded every 5 minutes or 30 minutes or 4 hours forexample—just as it was historically recorded on the paper record. Theremainder of the response data must be manually entered into the recordand thus suffers from the same limitations as noted above for themanually entered dose events. As a result, with current EMRs the doseand response data cannot be temporally correlated with any accuracy,vastly reducing the analytical and predictive value of the electronicdatabase and record. The systems described herein provide a technicalsolution to a technical problem. Furthermore, the benefits achieved bythe technical solutions of this disclosure exceed what can beaccomplished by manual processes.

In some examples, the automated data consolidation module 100,200 ofthis disclosure includes systems and methods for constructing granular(beat-by-beat, second-by-second) anesthetic, surgical and patientrecords that include both “dose” events—the things that are given ordone to the patient (inputs from medication injection and fluidmonitors, various support equipment and machine vision “observations”for example) and “response” events (inputs from electronic physiologicmonitors, various measurement devices and machine vision “observations”for example). In some examples, the automated data consolidation module100,200 of this disclosure automatically enters both dose and responseevents into the electronic record. In some examples, the dose andresponse events of this disclosure are continually recorded, creating avery granular record. Electronic memory has gotten so inexpensive thatvast amounts of data can be practically recorded for analysis at a latertime. In some examples, the dose and response event data of thisdisclosure that do not change very fast or frequently, may becontinually recorded by the processing circuitry 157,257 butintermittently recorded in the EMR to reduce the size of the record.

In some examples, the automated data consolidation module 100,200 ofthis disclosure automatically enters both dose and response events intothe electronic record and temporally correlates the dose and responseevents when they are recorded. Temporally correlating the dose andresponse data can be accomplished by recording dose and response data inparallel, adding integrated or external time stamps to the dose andresponse data or any other suitable method of temporally matching thedata sets.

In an example, the module 201 of FIG. 2, can include a housing 299having a lower section 299A and a tower-like upper section 299B, whereinthe lower section 299A can be configured to house unrelated wasteheat-producing electronic and electromechanical surgical equipment, andwherein the tower-like upper section 299B can be located on top of thelower section 299A. The module 201 can also include a cowling 299Cexternal or internal to the housing 299 that substantially confineswaste heat generated by the unrelated waste heat-producing electronicand electromechanical medical equipment. In some examples, theelectronic and electromechanical medical equipment may be mounted on themodule 201,101. In some examples, the electronic and electromechanicalmedical equipment may be in electrical communication with the module201,101. In some examples, the electronic and electromechanical medicalequipment may be wireless communication with the module 201,101.

In some examples, most of the electronic and electromechanical medicalequipment that are in, on or connected to the module are eitherparticipants in dose events (something given to or done to the patient)or participants in response events (measuring or observing the patient'sresponse). The physical location of the electronic and electromechanicalmedical equipment in or on the automated data consolidation module100,200 of this disclosure, makes collecting data from the variousequipment very efficient.

Unfortunately, most medical equipment available today does not produce adigital output signal that reports its operating settings andmeasurements in a recordable format. In some examples, the automateddata consolidation module 100,200 allows rules to be applied to thevarious medical equipment that is housed within the module, mounted onthe module, or is in electrical communication with or in wirelesscommunication with the module. For example, the hospital or owner of theautomated data consolidation module 100,200 could require that that allequipment in or on the module produce digital data reflecting theequipment's operating parameters and sensor inputs. Equipmentmanufacturers would have a business interest in adding digital outputsto their equipment if they want to sell their equipment to thathospital.

At this date there are few data standards for medical equipment and eachmanufacturer uses any data format that they choose. In some examples,the hospital or the owner of the “big data” database or the owner of theautomated data consolidation module 100,200 could require that that allequipment in or on the module produce data in prescribed data standardsand protocols so that translation of the data is not necessary. In someexamples, the data standards would document the expectations for one ormore of: format, definition, structuring, tagging, transmission,manipulation, use and management of the data. In some examples, the datastandard defines entity names, data element names, descriptions,definitions, and formatting rules. Translating data from one datastandard to another introduces the possibility of error. Therefore,producing data for input to the processing circuitry 157,257 in theprescribed standards and formats minimizes the opportunity for error byavoiding data translation. However, in some examples, the processingcircuitry 157,257 of the automated data consolidation module 100,200includes software for translating data when necessary.

In some examples, the hospital or the owner of the “big data” databaseor the owner of the automated data consolidation module 100,200 couldrequire that all equipment in or on the module produce data to aprescribed data model that includes all relavent input record fields forthat specific type of equipment. When data is used in “big data”databases, it is very common that there are missing data. The automateddata consolidation module 100,200 of this disclosure can minimizemissing data by using prescribed data models for each type of equipmentand requiring that the data fields be filled. In some examples, the datastandard or data model also prescribe the format of the data in eachdata field. Labeling the fields with consistent labels helps when AIanalysis is done on the “big data.”

In some examples, the hospital or the owner of the “big data” databaseor the owner of the automated data consolidation module 100,200 couldrequire that that all equipment in or on the module produce datainstantly. In some examples, instant data production is desirable sothat the processing circuitry 157,257 can apply a time stamp or othertime designation to the data so that the data can be temporallycorrelated with other data. Instant data production also allows streamprocessing to be performed on the data for real-time feedback.Automating data input is far superior to manual data input for manyreasons. One of the important reasons is that automated input can acceptand record high volume, instantaneous data production allowing temporalcorrelations between dose event data and response event data.

In some examples, consistent data standards, consistent data formatting,consistent data fields, consistent data labeling and time stamping,produces data for a “big data” database that is most usable for AI andML analysis and research.

In some examples, consistent data standards, consistent data formatting,consistent data fields also produce data that is more protectable fromthe cyber security point of view. Having inputted structured andsemi-structured data with consistent data standards, consistent dataformatting, consistent data fields helps the processing circuitry157,257 recognize inputted data that could be viral or malicious. Insome examples, the processing circuitry 157,257 and software of theautomated data consolidation module 100,200 can provide front lineaccess control and firewall protection, preventing corrupt data fromentering the system and being transferred to either the EMR or the “bigdata” database. In some examples, the processing circuitry 157,257 andsoftware include one or more of access control software, firewallsoftware, antivirus software, anti-malware, anti-spyware, anti-tampersoftware, anti-subversion software and other cyber security software.

If the thousands of monitors and other pieces of medical equipment in ahospital inputted their data directly to the EMR, it would be verychallenging to provide cyber security on all of those inputssimultaneously. In some examples, since the automated data consolidationmodule 100,200 is receiving data inputs from only 2-30 pieces of medicalequipment and sensors, cyber security is much more manageable.Therefore, it may be advantageous to use the processing circuitry157,257 and software of the automated data consolidation module 100,200to provide cyber security on the inputted data. When the inputted datais known to be secure, it can be freely transferred to the EMR or “bigdata” database without fear of corrupting the system.

In some examples, the hospital or the owner of the “big data” databaseor the owner of the automated data consolidation module 100,200 couldrequire that that all equipment in or on the module produce datacontinuously. One of the “V's” of “big data” is volume—the more data thebetter. Therefore, in some examples, data and measurements that arechanging may be recorded continuously to produce the most data “volume”for that data set. In some examples, the processing circuitry 157,257and software can do data “filtering” in the presence of large size datato discard information that is not useful for healthcare monitoringbased on defined criteria. This may include for example, intermittentlyrecording data that changes slowly such as the patient's temperature,rather than continuously recording. In some examples, when the data isnot changing or changing very slowly, for example the settings on apiece of equipment or the patient's temperature, the processingcircuitry 157,257 may record the data into the EHR or “big data”database, intermittently for efficiency, without losing any “volume.”

In some examples, the processing circuitry 157,257 and software can dodata “cleaning” such as normalization, noise reduction and missing datamanagement. “Sensor fusion” is a technique that may be utilized tosimultaneously analyze data from multiple sensors, in order to detectand discard erroneous data from a single sensor. In some examples, theprocessing circuitry 157,257 and software can be used in many other waysto cleanse, organize and prepare the input data. Data “filtering” anddata “cleaning” prepare the data to enhance the reliability of datamining techniques. Processing raw data without preparation routines mayrequire extra computational resources that are not affordable in a “bigdata” context.

In some examples, the processing circuitry 157,257 and software execute“stream processing” for applications requiring real-time feedback. Insome examples, streaming data analytics in healthcare can be defined asa systematic use of continuous waveform and related medical recordinformation developed through applied analytical disciplines, to drivedecision making for the patient care. In some examples, as shown in FIG.25, streaming healthcare data 2500 may start with fast data ingestion2502 which may include continuous waveforms such as EKG or pulseoximetry or other non-waveform data such as blood pressure 2504. Thenext step in the process is situational and contextual awareness 2506,where the processing circuitry 157,257 and software correlate and enrichthe data set with data from the EHR, patient history, labs, allergiesand medications and other data 2508. The next step in the process issignal processing and feature extraction 2510, where advanced analytics2512 consume relevant data to produce insights. The final step in theprocess is producing actionable insight 2514, which can include clinicaldecision support and alarms 2516. In some examples, it is advantageousto regulate the access of incoming data with access control and firewallsoftware 2518.

In some examples, when the objective is to deliver data to a “big data”database, the data must be pooled. Data in the “raw” state needs to beprocessed or transformed. In a service-oriented architectural approach,the data may stay raw and services may be used to call, retrieve andprocess the data. In the data warehousing approach, data from varioussources is aggregated and made ready for processing, although the datais not available in real-time. The steps of extract, transform, and load(ETL) can be used to cleans and ready data from diverse sources. In someexamples, it is most efficient for the processing circuitry 157,257 andsoftware of the automated data consolidation module 100,200 of thisdisclosure, to process and transform the raw data before sending it tothe “big data” database.

In some examples, an applied conceptual architecture of big dataanalytics 2600 is shown in FIG. 26. Data from multiple sources 2602 canbe inputted into the processing circuitry 157,257 and software of theautomated data consolidation module 100,200. Big Data sources 2602 anddata types include but are not limited to: Machine to machinedata—readings from remote sensors, meters and other vital signs devices;big transaction data—health care claims and other billing records;biometric data—x-ray and other medical images, blood pressure, pulse andpulse-oximetry readings, and other similar types of data;human-generated data—unstructured and semi-structured data such as EMRsand physician notes. In some examples, it is advantageous to regulatethe access of incoming data with access control and firewall software2614.

In some examples, the processing circuitry 157,257 and software of theautomated data consolidation module 100,200 take the data and performthe steps of extract, transform, and load (ETL) 2604 to cleans and readythe data from diverse Big Data sources 2602. Once the data is processedor transformed, several options are available for proceeding with bigdata. A service-oriented architectural approach combined with webservices (middleware) is one possibility 2606. The data stays raw andservices are used to call, retrieve and process the data. Anotherapproach is data warehousing 2608 wherein data from various sources isaggregated and made ready for processing. With data warehousing 2608 thedata is not available in real-time.

Data transformation 2604 via the steps of extract, transform, and load(ETL), cleanses and readies the data from diverse sources 2602.Depending on whether the data is structured or unstructured, severaldata formats can be inputted to a big data analytics platform 2610. Bigdata is then organized into big data analytics applications 2612 such asqueries, reports, OLAP and data mining, so that it can be useful.

In some examples, with “big data” database data, the processingcircuitry 157,257 and software may execute “batch processing,” analyzingand processing the data over a specified period of time. Batchprocessing aims to process a high volume of data by collecting andstoring batches to be analyzed in order to generate results. In someexamples, the processing circuitry 157,257 and software can serve as a“node” in batch computing, where big data is split into small piecesthat are distributed to multiple nodes in order to obtain intermediateresults. Once data processing by nodes is terminated, outcomes will beaggregated in order to generate the final results. In some examples,stream processing and batch processing may be both used, either inparallel or sequentially.

In some examples, the automatically entered, temporally correlated doseand response events in the patient's electronic medical record (EMR) maybe analyzed by artificial intelligence (AI) and/or machine learning (ML)software in the processing circuitry 157,257 of the module for immediateadvisory, alerts and feedback to the clinician. The patient's own datacan be analyzed by AI algorithms that were derived from general medicalknowledge and research or may be analyzed by AI algorithms that werederived from AI analysis of the pooled database of many patients, suchas machine learning (ML) or a combination of the two.

In some examples, the automatically entered, temporally correlated doseand response events in the patient's electronic record may be pooledwith the records of other patients in a database that can be analyzedwith artificial intelligence and machine learning software stored on amemory and executed by processing circuitry. This analysis can occur atthe time of temporal correlation or at a later time. This type ofanalysis is sometimes known as “big data.” The pooled data (e.g.aggregated data, compiled data, such as collective data of a group ofpatients, collective data for a specific patient over different periodsof time, doses, responses, medications, providers) may be stored onproprietary servers or may be stored in “the cloud.” In any event, the“big data” database of this disclosure is complete and granular(compared to current healthcare databases) because it was automaticallyentered and at least some of the data is continually recorded. The “bigdata” database of this disclosure includes both dose events and responseevents (compared to current healthcare databases that are weightedtoward response events) and the dose and response events are temporallycorrelated (compared to current healthcare databases). Temporalcorrelation can include time matching different sets of data streams.

“Big data” has been characterized by “four Vs”: Volume, Velocity,Variety and Veracity. In some examples, the automated data consolidationmodule 100,200 of this disclosure addresses each of thesecharacteristics in order to optimize the acquired data for “big data”use and analysis.

Volume—The more data you have the better it is. The automated dataconsolidation module 100,200 of this disclosure maximizes data volume bycollecting data not only from the physiologic monitors but also all ofthe OR equipment and other dose events including machine vision videoobservations. Further, the automated data consolidation module 100,200can provide either continuous recordings or intermittent recordingsdepending on the speed of change in the data and the usefulness ofcontinuous data streams for any given data set or field.

Velocity—The data is accumulated in real-time and at a rapid pace. Theautomated data consolidation module 100,200 of this disclosure maximizesdata velocity by automatically recording everything which removes manualdata entry that slows the data acquisition process down. The ability toperform real-time analytics against such high-volume data in motioncould revolutionize healthcare.

Variety—How different is the data. The automated data consolidationmodule 100,200 of this disclosure maximizes data variety by recordingmost if not all structured and semi-structured data sources in the OR orother care locations, including but not limited to: physiologicmonitors, equipment, medication injections, dose events, responseevents, fluids and video with artificial intelligence (AI) and/ormachine learning (ML) analysis of the patient and delivered care. Insome examples, the automated data consolidation module 100,200 may alsoaccept unstructured textual data entered by a keyboard, natural languagevoice recognition, from the EMR/EHR or other suitable source.

Veracity—How accurate is the data, the truthfulness of the data. Theautomated data consolidation module 100,200 of this disclosure maximizesdata veracity by automatically recording the data without requiringhuman input. In some examples, the automated data consolidation module100,200 of this disclosure includes Sensor Fusion, a data analyticsystem that monitors input data in order to reject clearly mistakeninputs such as might occur if an EKG lead comes lose forexample—improving the veracity of the database. Sensor Fusion analyzesthe input data from multiple sensors simultaneously in order to detectand reject inputs that are clearly erroneous. In some examples, SensorFusion may include a threshold-controlled artifact-removal module and/ora Kalman filter using statistical analysis to compare the input datastreams.

In some examples, the automated data input feature of the automated dataconsolidation module 100,200 is particularly important because theincreased variety of data input and high velocity of data input hinderthe ability to cleanse data before analyzing it and making decisions,magnifying the issue of data “trust.” The mere fact that the input isautomatic decreases the opportunity for human error.

In some examples as shown in FIGS. 1 and 2, module 101,201 of theautomated data consolidation module 100,200 may consolidate a widevariety of equipment used for anesthesia and surgery. Some of thisequipment, including but not limited to: the physiologic monitors, theurine output monitor and the blood loss monitor and video observation ofthe patient with AI and/or ML analysis to produce response event datathat can be automatically recorded by the automated data consolidationmodule 100,200.

In some examples as shown in FIGS. 1 and 2, module 101,201 of theautomated data consolidation module 100,200 may consolidate a widevariety of equipment that produce dose event data, including but notlimited to: medication identification and measurement system 128,228 andthe IV fluid identification and measurement system 130,230, gas flowmeters (not shown), the mechanical ventilator (not shown), thepneumoperitoneum insufflator (not shown), the electrosurgical generator(not shown) and video observation of the patient and provider with AIand/or ML analysis to produce dose event data that can be automaticallyrecorded by the automated data consolidation module 110,200 (e.g., doseevent data can be received by processing circuitry 157, 257 and storedon one or more storage devices).

The automated data consolidation module 100,200 not only physicallyconsolidates various electrical and electromechanical equipment but alsoconsolidates the data outputs of the various equipment for easyelectrical communication with the processing circuitry 157, 257. Theconsolidation of equipment in and on the module 101,201 of the automateddata consolidation module 100,200 solves another practical obstaclecurrently preventing an automated dose/response record. Currently, muchof the equipment mentioned above does not produce digital data thatdocuments the equipment's operation because the various equipmentmanufacturers have chosen not to provide output data. The practicalsolution is that the provider of the automated data consolidation module100,200 can require that any equipment that is to be located in or onthe module 101,201, must include data outputs and the provider can alsoprescribe the digital standard and format so that the data can beorganized efficiently and completely in a “big data” database. Thesebenefits are not provided in any conventional system.

In some examples, it is anticipated that the automated dataconsolidation module 100,200 can also accept input data from othermedical equipment and data sources by either wired or wirelessconnections. As with the equipment housed within or on the automateddata consolidation module 100,200, it is preferable if the input datafollows the digital standard and format applied to the other equipment.However, in some examples the processing circuitry 157, 257 cantranslate input data that is not in the desired digital standard orformat.

Machine vision cameras and software are very good at measuringdistances, movements, sizes, looking for defects, fluid levels, precisecolors and many other quality measurements of manufactured products.Machine vision cameras and software can also be “taught” through AI andML to analyze complex and rapidly evolving scenes such as those in frontof a car driving down the road. Machine vision cameras and softwaredon't get bored or distracted.

FIG. 12 illustrates an isometric view of another example module 1201including an automated data consolidation module 1200 for generating anautomated electronic anesthetic record located proximate to a patient.

In some examples as shown in FIG. 12, the automated data consolidationmodule 1200 of this disclosure includes systems and methods for usingmachine vision cameras 1299A,B and software to “observe” the patient. Ifthe patient is in surgery, the patient's head may be the focus of theobservation. In some examples, during surgery the machine vision cameras1299A,B and software may be “looking” for dose events including but notlimited to mask ventilation, endotracheal intubation or airwaysuctioning. In other words, the machine vision cameras 1299A,B cangenerate and send dose event information to be received and analyzed byprocessing circuitry 157, 257.

In some examples, during surgery the machine vision cameras 1299A,B andsoftware may be “looking” for (e.g., monitoring, sensing, detecting)response events including but not limited to: movement, grimacing,tearing or coughing or changes in skin color or sweating. In otherwords, the machine vision cameras 1299A,B can generate and send responseevent information to be received and analyzed by processing circuitry157, 257. Movement, grimacing, tearing and coughing are all signs of“light” anesthesia and that the patient may be in pain. Subtle changesin skin color from pinkish to blueish or grayish may indicate inadequateoxygenation or low perfusion. Subtle changes in skin color from pinkishto reddish may indicate hyperthermia or an allergic reaction. Sweatingmay indicate inadequate perfusion or hyperthermia. In any event,observing the patient is regarded by the American Society ofAnesthesiologists as the most basic of all monitors. Machine visioncameras 1299A,B and software do not get bored and can be programed torecognize subtle changes that may be missed by the human observer. Bothmovements and skin color changes may be subtle. The baseline color maybe very difficult for the observer to remember, whereas the machinevision cameras 1299A,B and software can precisely “remember” thebaseline color and recognize even subtle changes over time. In someexamples, pain may be detected by facial expression analysis, such as byprocessing circuitry 1257 receiving response event information from themachine vision cameras 1299A,B, analyzing such response eventinformation, and recording or displaying the information, or alerting aclinician or other personnel via a user interface, such as the display1226 or the remote display 1287, shown in FIG. 12.

In some examples, vital signs such as heart rate, respiration rate,blood oxygen saturation and temperature can be measured remotely viacamera-based methods. Vital signs can be extracted from the opticaldetection of blood-induced skin color variations—remotephotoplethysmography (rPPG). In some examples, this technique works bestusing visible light and Red Green Blue (RBG) cameras. However, it hasalso been shown to work using near-infrared light (NIR) and NIR cameras.Just like pulse oximetry, a well-known technology for detecting pulseand blood oxygen saturation, rPPG relies on the varying absorption ofdifferent wavelengths of light caused by blood volume changes and theoxygen saturation of the blood in the small blood vessels beneath theskin. Unlike pulse oximetry that needs to be in contact with the skin,rPPG enables contactless monitoring of human cardiac activities bydetecting the pulse-induced subtle color changes on the human skinsurface using a regular RBG camera.

In some examples, this disclosure anticipates automating the remotephotoplethysmography using techniques such as, but not limited to,region of interest (RoI) detection and full video pulse extraction(FVP). In some examples, this disclosure also anticipates using acombination of visible light and near-infrared light wavelengths todetect different parameters. In some examples, additional measurementsare anticipated using the rPPG technology including but not limited tomeasuring blood glucose and hemoglobin levels.

FIG. 13 illustrates an isometric view of yet another example module 1301including an automated data consolidation module 1300 for generating anautomated electronic record located proximate to a patient.

In some examples as shown in FIG. 13, if the patient is on the ward, inthe nursing home or other long-term care facility or at home, the wholepatient may be the focus of the observation. In some examples, if thepatient is on the ward, in the nursing home or other long-term carefacility, or at home, the machine vision cameras 1399 and software maybe “looking” for (e.g., sensing, monitoring, detecting) dose eventsincluding but not limited to repositioning the patient or suctioning theairway, feeding and eating, or assisting the patient out of bed or anyother nursing procedure.

In some examples, if the patient is on the ward, in the nursing home orother long-term care facility, or at home, the machine vision cameras1399 and software may be “looking” (e.g., sensing, monitoring,detecting) for response events including but not limited to restlessnessor getting out of bed without assistance, coughing or changes in thebreathing pattern or changes in skin color. Recent research has shownthat machine vision cameras 1399 and software with AI can recognizemoods with reasonable accuracy. The patient's mood could be an importantresponse event that should be recorded.

As shown in technique 2400 of FIG. 24, various operations can beperformed by the dose-response systems 100, 200, 1200, 1300 andsubsystems disclosed herein. While the technique 2400 can be performedby the dose-response systems 100, 200, 1200, 1300, the technique 2400can also be performed by other systems, and the systems 100, 200, 1200,1300 can perform other techniques. In an example, a dose-response systemcan perform technique 2400 using processing circuitry and one or morestorage devices including a memory. The memory can have instructionsstored thereon, which when executed by the processing circuitry causethe processing circuitry to perform the operations. For example,operation 2402 can include receiving dose information (or dose event)from one or more sensors. Operation 2404 can include receiving responseinformation (or response event) from one or more sensors. The doseinformation can include data corresponding to a dose provided to thepatient, and the response information can include data corresponding toa response of the patient. Operation 2406 can include temporallycorrelating the dose information and the response information. Operation2408 can include saving the temporally correlated dose-responseinformation to at least one of the one or more storage devices such asstorage device 1516 of FIG. 15. Temporally correlating dose-responseinformation can include continuously temporally correlating thedose-response information, such as in a processor -based timer, in abeat-by-beat, second-by-second type manner, as opposed to the extendedintervals and less accurate methods of traditional medical practice. Insome examples, temporally correlating the dose-response informationincludes temporally correlating the dose-response information, such asin a range between every 0.1 seconds to 5 minutes. In some examples, itmay be preferable to temporally correlate in a range between every 0.1seconds to 1 minute. It may be more preferable to temporally correlatein a range between every 1 second to 1 minute, however, any suitabletemporal correlation may be used to facilitate accurate, safe andmeaningful correlation where a direct, time-based relationship betweenthe dose and the response is captured.

In some examples, operation 2410 can include displaying informationcorresponding to the dose-response data. In operation 2412, if theprocessing circuitry detects a concerning response, the processingcircuitry can alert a user to the response. Operation 2414 can includeanalyzing the temporally correlated data. In some examples, operation2414 can include aggregating the temporally correlated dose-responseinformation with temporally correlated dose-response informationcollected from other patients. In some examples, operation 2414 caninclude aggregating the temporally correlated dose-response informationwith temporally correlated dose-response information collected from thesame patient over a different period of time. Comparing data from thesame patient may allow a provider to detect when a patient develops atolerance to a drug, such as a pain killer. In an example, if the rateof grimacing by the patient increases or they start to grimace sooner(e.g., response) after being given a pain medication (dose), as the painmedication is given over different days, weeks and months. Operation2414 can also facilitate machine learning.

In technique 2400, the dose information can include data/eventsgenerated by one or more of: an RFID interrogator, a barcode reader, aQR reader, a machine vision digital camera, any combination thereof, orany other suitable sensor. The response information can includedata/events generated by a machine digital camera or any other suitablesensor. The response information can include an image of one or moremovements, secretions or skin color changes, and the processingcircuitry can be configured to identify changes in the responseinformation. In some examples the processing circuitry can use machinelearning to identify changes in the response information. In someexamples, the response information includes physiologic data generatedby a physiologic monitor. For example, the physiologic data can includeat least one of: electrocardiogram, pulse oximetry, blood pressure,temperature, end-tidal CO2, expired gases, respiratory rate, hemoglobin,hematocrit, cardiac output, central venous pressure, pulmonary arterypressure, brain activity monitor, sedation monitor, urine output, bloodloss, blood electrolytes, blood glucose, blood coagulability,train-of-four relaxation monitor data, IV extravasation monitor data andbody weight, and any combination thereof or other suitable physiologicdata.

Implementation of any of the techniques described herein may be done invarious ways. For example, these techniques may be implemented inhardware, software, or a combination thereof. For a hardwareimplementation, the processing units may be implemented within on ormore application specific integrated circuits (ASICs), digital signalprocessors (DSPs), digital signal processing devices (DSPDs),programmable logic devices (PLDs), field programmable gate arrays(FPGAs), processors, controllers, micro-controllers, microprocessors,other electronic units designed to perform the functions describedabove, and/or a combination thereof.

Furthermore, embodiments may be implemented by hardware, software,scripting languages, firmware, middleware, microcode, hardwaredescription languages, and/or any combination thereof. When implementedin software, firmware, middleware, scripting language, and/or microcode,the program code or code segments to perform the necessary tasks may bestored in a machine-readable medium such as a storage medium. A codesegment or machine-executable instruction may represent a procedure, afunction, a subprogram, a program, a routine, a subroutine, a module, asoftware package, a script, a class, or any combination of instructions,data structures, and/or program statements. A code segment may becoupled to another code segment or a hardware circuit by passing and/orreceiving information, data, arguments, parameters, and/or memorycontents. Information, arguments, parameters, data, etc. may be passed,forwarded, or transmitted via any suitable means including memorysharing, message passing, token passing, network transmission, etc.

For a firmware and/or software implementation, the techniques may beimplemented with modules (e.g., procedures, functions, and so on) thatperform the functions described herein. Any machine-readable mediumtangibly embodying instructions may be used in implementing thetechniques described herein. For example, software codes may be storedin a memory. Memory may be implemented within the processor or externalto the processor. As used herein the term “memory” refers to any type oflong term, short term, volatile, nonvolatile, or other storage mediumand is not to be limited to any particular type of memory or number ofmemories, or type of media upon which memory is stored.

There are two common modes for ML: supervised ML and unsupervised ML.Supervised ML uses prior knowledge (e.g., examples that correlate inputsto outputs or outcomes) to learn the relationships between the inputsand the outputs. The goal of supervised ML is to learn a function that,given some training data, best approximates the relationship between thetraining inputs and outputs so that the ML model can implement the samerelationships when given inputs to generate the corresponding outputs.Unsupervised ML is the training of an ML algorithm using informationthat is neither classified nor labeled, and allowing the algorithm toact on that information without guidance. Unsupervised ML is useful inexploratory analysis because it can automatically identify structure indata. Either of supervised ML or unsupervised ML can be used to trainthe systems described herein to correlate information from one or moresensors, such as a machine vision digital camera as having a particularmeaning. For example, either of supervised or unsupervised ML can beused to train the systems to correlate facial expressions with responseevents. The system can be trained to “read” a particular patient, or agroup of patients that may or may not include the particular patientbeing monitored.

Common tasks for supervised ML are classification problems andregression problems. Classification problems, also referred to ascategorization problems, aim at classifying items into one of severalcategory values (for example, is this object an apple or an orange?).Regression algorithms aim at quantifying some items (for example, byproviding a score to the value of some input). Some examples of commonlyused supervised-ML algorithms are Logistic Regression (LR), Naive-Bayes,Random Forest (RF), neural networks (NN), deep neural networks (DNN),matrix factorization, and Support Vector Machines (SVM). n

Some common tasks for unsupervised ML include clustering, representationlearning, and density estimation. Some examples of commonly usedunsupervised-ML algorithms are K-means clustering, principal componentanalysis, and autoencoders.

FIG. 15 illustrates an example electronic and/or electromechanicalsystem 1500 of a medical system in accordance with some examplesdescribed herein. The system 1500 will be described with respect to theautomated data consolidation module 100, 200, but can include any of thefeatures described herein to perform any of the methods or techniquesdescribed herein, for example, by using the processor 1502. Theprocessor can include processing circuitry 157 or 257 of FIGS. 1 and 2.In some examples, the processing circuitry 1502 can include but is notlimited to, electronic circuits, a control module processing circuitryand/or a processor. The processing circuitry may be in communicationwith one or more memory and one or more storage devices. A singleprocessor can coordinate and control multiple, or even all the aspectsof the system 1500 (e.g., of modules 101, 201), or multiple processorscan control all the aspects of the system 1500. In some examples thestorage device 1516 or memory 1504, 1506, 1516 can include at least aportion of the patient's anesthetic record saved thereon. The system1500 can also include any of the circuitry and electronic and/orelectromechanical components described herein, including but not limitedto, any of the sensor(s) described herein (e.g., sensors 1521), such asbut not limited to, RFID barcode or QR codes sensors, machine visioncameras, retinal scanners, facial recognition scanners, fingerprintreaders, actuators and position sensors described herein. The system1500 may also include or interface with accessories or other featuressuch as any of: a remote display or wireless tablet (e.g., 287, FIG. 2),as well as any of the other systems described herein.

The processing circuitry 1502 can receive information from the varioussensors described herein, make various determinations based on theinformation from the sensors, output the information or determinationsfrom the information for output on the display or wireless tablet,output instructions to provide an alert or an alarm, power variouscomponents, actuate actuators such as clamps and flow managing devices,etc., or alert another system or user, as described herein. For the sakeof brevity, select systems and combinations are described in furtherdetail above and in the example sets provided in the Notes and VariousExamples section below. Other embodiments are possible and within thescope of this disclosure.

Further, FIG. 15 illustrates generally an example of a block diagram ofa machine (e.g., of module 101, 201) upon which any one or more of thetechniques (e.g., methodologies) discussed herein may be performed inaccordance with some embodiments. In alternative embodiments, themachine 1500 may operate as a standalone device or may be connected(e.g., networked) to other machines. In a networked deployment, themachine 1500 may operate in the capacity of a server machine, a clientmachine, or both in server-client network environments. The machine1500, or portions thereof may include a personal computer (PC), a tabletPC, a personal digital assistant (PDA), a mobile telephone, a webappliance, a network router, switch or bridge, or any machine capable ofexecuting instructions (sequential or otherwise) that specify actions tobe taken by that machine. Further, while only a single machine isillustrated, the term “machine” shall also be taken to include anycollection of machines that individually or jointly execute a set (ormultiple sets) of instructions to perform any one or more of themethodologies discussed herein, such as cloud computing, software as aservice (SaaS), other computer cluster configurations.

Examples, as described herein, may include, or may operate on, logic ora number of components, modules, or like mechanisms. Such mechanisms aretangible entities (e.g., hardware) capable of performing specifiedoperations when operating. In an example, the hardware may bespecifically configured to carry out a specific operation (e.g.,hardwired). In an example, the hardware may include configurableexecution units (e.g., transistors, circuits, etc.) and a computerreadable medium containing instructions, where the instructionsconfigure the execution units to carry out a specific operation when inoperation. The configuring may occur under the direction of theexecution units or a loading mechanism. Accordingly, the execution unitsare communicatively coupled to the computer readable medium when thedevice is operating. For example, under operation, the execution unitsmay be configured by a first set of instructions to implement a firstset of features at one point in time and reconfigured by a second set ofinstructions to implement a second set of features.

Machine (e.g., computer system) 1500 may include a hardware processor1502 (e.g., processing circuitry 157, 257, a central processing unit(CPU), a graphics processing unit (GPU), a hardware processor core, orany combination thereof), a main memory 1504 and a static memory 1506,some or all of which may communicate with each other via an interlink(e.g., bus) 1508. The machine 1500 may further include a display unit1510, an alphanumeric input device 1512 (e.g., a keyboard), and a userinterface (UI) navigation device 1514 (e.g., a mouse). In an example,the display device 1510, an input device such as an alphanumeric inputdevice 1512 and UI navigation device 1514 may be a touch screen display.The display unit 1510 may include goggles, glasses, or other AR or VRdisplay components. For example, the display unit may be worn on a headof a user and may provide a heads-up-display to the user. Thealphanumeric input device 1512 may include a virtual keyboard (e.g., akeyboard displayed virtually in a VR or AR setting.

The machine 1500 may additionally include a storage device (e.g., driveunit) 1516, a signal generation device 1518 (e.g., a speaker), a networkinterface device 1520, and one or more sensors 1521, such as a globalpositioning system (GPS) sensor, compass, accelerometer, or othersensor. The machine 1500 may include an output controller 1528, such asa serial (e.g., universal serial bus (USB), parallel, or other wired orwireless (e.g., infrared (IR), near field communication (NFC), etc.)connection to communicate or control one or more peripheral devices oractuators of the system. Peripheral devices can include but are notlimited to any displays, controllers or memories in electricalcommunication with the system, and actuators can include but are notlimited to: one or more stop-flow clamps 1060A,B (FIG. 10) and one ormore flow rate clamps 1478 (FIG. 14) of the system.

The storage device 1516 may include a machine readable medium 1522 thatis non-transitory on which is stored one or more sets of data structuresor instructions 1524 (e.g., software) embodying or utilized by any oneor more of the techniques or functions described herein. Theinstructions 1524 may also reside, completely or at least partially,within the main memory 1504, within static memory 1506, or within thehardware processor 1502 during execution thereof by the machine 1500. Inan example, one or any combination of the hardware processor 1502, themain memory 1504, the static memory 1506, or the storage device 1516 mayconstitute machine readable media that may be non-transitory.

While the machine readable medium 1522 is illustrated as a singlemedium, the term “machine readable medium” may include a single mediumor multiple media (e.g., a centralized or distributed database, orassociated caches and servers) configured to store the one or moreinstructions 1524.

The term “machine readable medium” may include any medium that iscapable of storing, encoding, or carrying instructions for execution bythe machine 1500 and that cause the machine 1500 to perform any one ormore of the techniques of the present disclosure, or that is capable ofstoring, encoding or carrying data structures used by or associated withsuch instructions. Non-limiting machine-readable medium examples mayinclude solid-state memories, and optical and magnetic media. Specificexamples of machine readable media may include: non-volatile memory,such as semiconductor memory devices (e.g., Electrically ProgrammableRead-Only Memory (EPROM), Electrically Erasable Programmable Read-OnlyMemory (EEPROM)) and flash memory devices; magnetic disks, such asinternal hard disks and removable disks; magneto-optical disks; andCD-ROM and DVD-ROM disks.

The instructions 1524 may further be transmitted or received over acommunications network 1526 using a transmission medium via the networkinterface device 1520 utilizing any one of a number of transferprotocols (e.g., frame relay, internet protocol (IP), transmissioncontrol protocol (TCP), user datagram protocol (UDP), hypertext transferprotocol (HTTP), etc.). Example communication networks may include alocal area network (LAN), a wide area network (WAN), a packet datanetwork (e.g., the Internet), mobile telephone networks (e.g., cellularnetworks), Plain Old Telephone (POTS) networks, and wireless datanetworks (e.g., Institute of Electrical and Electronics Engineers (IEEE)802.11 family of standards known as Wi-Fi®, IEEE 802.16 family ofstandards known as WiMax®), IEEE 802.15.4 family of standards,peer-to-peer (P2P) networks, among others. In an example, the networkinterface device 1520 may include one or more physical jacks (e.g.,Ethernet, coaxial, or phone jacks) or one or more antennas to connect tothe communications network 1526. In an example, the network interfacedevice 1520 may include a plurality of antennas to wirelesslycommunicate using at least one of single-input multiple-output (SIMO),multiple-input multiple-output (MIMO), or multiple-input single-output(MISO) techniques. The term “transmission medium” shall be taken toinclude any intangible medium that is capable of storing, encoding orcarrying instructions for execution by the machine 1500, and includesdigital or analog communications signals or other intangible medium tofacilitate communication of such software.

Method examples described herein may be machine or computer-implementedat least in part. Some examples may include a computer-readable mediumor machine-readable medium encoded with instructions operable toconfigure an electronic device to perform methods as described in theexamples. An implementation of such methods may include code, such asmicrocode, assembly language code, a higher-level language code, or thelike. Such code may include computer readable instructions forperforming various methods. The code may form portions of computerprogram products. Further, in an example, the code may be tangiblystored on one or more volatile, non-transitory, or non-volatile tangiblecomputer-readable media, such as during execution or at other times.Examples of these tangible computer-readable media may include, but arenot limited to, hard disks, removable magnetic disks, removable opticaldisks (e.g., compact disks and digital video disks), magnetic cassettes,memory cards or sticks, random access memories (RAMs), read onlymemories (ROMs), and the like.

Some of the benefits of the automated data consolidation module 100, 200of FIGS. 1 and 2 and the subsystems described throughout thisdisclosure, and including the machine 1500 (FIG. 15), can includefeatures to help with monitoring medication, fluid and anesthesiadelivery, as well as documenting medication, fluid and anesthesiadelivery, as well as other functions. In general, doctors and nurses arenot interested in replacing themselves and their jobs with automateddrug delivery or automated anesthesia systems. However, there is greatinterest in automated record keeping. Virtually all healthcare providerswould prefer the “old” paper record and a pen to the “new” computerrecords. Filling out the electronic medical record (EMR) using acomputer keyboard, mouse and various menus is widely viewed as a slow,cumbersome and distracting process. The challenge with automated recordkeeping is automating the data input that documents the numerousactivities, anesthesia related events, fluid, gas and medicationadministration, ventilator settings, pressure off-loading effectiveness,as well as outputs such as blood loss and urine output, that constitutean anesthetic experience.

A challenge in implementing the automated data consolidation module 100,200 with an automated electronic anesthetic record (EAR) or electronicmedical record (EMR) is to force as little change in the caregiver'sroutine as possible onto the clinicians using this system. Medicalpersonnel tend to be creatures of habit and tradition and they generallydo not like change. For example, IV medications are traditionallyadministered from a syringe and the dose is determined by the caregiverobserving the plunger moving relative to a scale printed on the syringe.Maintaining this general technique of drug administration may have thehighest probability of acceptance by healthcare users who are typicallyslow to embrace changes in their routine.

Further, with regard to benefits of the modules, systems and machinesdescribed herein, the automated data consolidation module 200 of module201 can generate an automated electronic medical record (EMR) with themodule 201 locatable proximate to the patient 202. The module 201 can bea module for housing unrelated electronic and electromechanical surgicalequipment. The module 201 need not necessarily be configured to houseunrelated electronic and electromechanical surgical equipment in allexamples, and other modules can include the system for generating anautomated EMR.

The module 201 can be an automated EMR system that can include one ormore systems (e.g., 200, 228, 230) configured to measure (e.g., monitor)and record one or more of functions involved in a surgical anestheticenvironment, and can include life support functions. The one or moresystems 228, 230 can measure and record data automatically. However, insome examples, a user may initiate any of the systems described hereinto measure and/or record data. These various measurements may beelectronically recorded (such as on mass storage 1516 (FIG. 15) anddisplayed on the electronic anesthetic record display 226 (e.g., displaydevice 1510, FIG. 15). Inputs to the automated EMR system may be managedby the anesthetic record input component 224 (e.g., input device 1512;FIG. 15). The anesthetic record input component 224 (e.g., input device1512; FIG. 15) can include a touch-screen display 226 that organizes allof the inputs to the EMR into easily accessed and utilized information.In some examples, and as shown in FIG. 2, the identification andmeasurement system 228 of this disclosure may be located proximate thepatient 202. The control displays for the identification and measurementsystem 228 may include a dedicated display proximate the identificationand measurement system 228 or may be shared space on the anestheticrecord input component 224 or display 226. In these locations, theinformation and controls of the identification and measurement system228 can be viewed by the anesthesiologist or other user, in a singlefield of vision with the patient and surgical field.

Example methods of employing the systems, modules and machines disclosedherein are described throughout this disclosure and in the methods ofFIGS. 16-21 which are illustrative in nature. Other methods describedherein may also be performed by the systems, modules and machinesdescribed herein, and the systems modules and machines described hereinmay be used to perform other methods.

FIGS. 16-18 show flow charts illustrating techniques for identification,measurement, monitoring, security and safety related to medicationsand/or IV fluids. The methods may be used with the systems, sub-systemsand modules of FIGS. 1-15 (e.g., 101, 201, 1500), but may also be usedwith other systems. Likewise, the systems, subsystems of modules ofFIGS. 1-15 may also be used with other methods. The techniques 1600,1700, 1800, 1900, 2000, 2100, 2300 can be performed by at least onenon-transitory machine-readable medium (e.g., computer readable)including instructions for operation of the systems described herein.Some steps of techniques may be performed by a provider. The systems caninclude processing circuitry (e.g., 157, 257, 1500, including one ormore processors, processing circuitry hardware) for executing theinstructions. The instructions, when executed by the processingcircuitry can cause the processing circuitry to perform operationsdescribed in FIGS. 16-21 and 23, and as described in the examplesthroughout this disclosure.

FIG. 16 is a flow chart illustrating an example technique 1600 of IVfluid identification and measurement. To start the technique, inoperation 1602 a provider hangs an IV fluid bag and attached dripchamber on electronic scale hooks in an IV fluid identification andmeasurement unit (e.g., FIG. 14). In operation 1604, a machine visioncamera and software can identify the fluid and bag by the barcode labelon the IV fluid bag. In operation 1606, the machine vision camera andsoftware can identify the individual drops in the drip chamber andmeasure the size of the drop to determine the fluid volume per drop andcount the number of drops per unit time. In operation 1608 the machinevision camera and software can calculate the flow rate by multiplyingthe number of drops per unit time by the volume/drop. In operation 1610,the fluid flow rate is displayed and document in the EMR.

In operation 1612, if the machine vision camera and software fails toidentify individual drops in the drip chamber, in operation 1614 themachine vision camera and software can look for a floating ball (e.g.,float) that is located in the drip chamber to determine if the ball isfloating and if the ball is moving. In operation 1616, when the ball isnot floating and/or moving, IV clamps are closed and the provider canchange the empty IV bag if necessary. In operation 1618, if the machinevision camera and software can determine that the ball is floating andmoving, the system determines that the fluid flow is so fast that thefluid flow is constant or continuous such that individual drops cannotbe measured. In operation 1618, because individual drops cannot bedetermined, the system switches to measuring the fluid flow rate usingan electronic IV scale (FIG. 14) to determine the fluid flow rate. Inoperation 1620, the fluid flow rate can be determined by monitoring thechange in IV bag weight per time. In operation 1622, the fluid flow ratecan be displayed and documented in the EMR.

FIG. 17 is a second flow chart illustrating a technique 1700 includingaspects of the technique 1600 of IV fluid identification and measurementfrom the perspective of processing circuitry (e.g., 257, FIG. 2; 1502,FIG. 15). The technique 1700 may include an operation 1702 to receive IVfluid identification information from a first IV sensor (e.g., one ormore sensors), such as an RFID or barcode reader to identify the fluidor other characteristics of an IV fluid bag as described herein.Operation 1704 can include saving the IV identification information to astorage device (e.g., one or more storage devices, memory, EMR).Operation 1706 can include to receive fluid drop information from asecond IV sensor, such as a machine vision camera that detects, sensesand measures an individual drop in a drip chamber to determine a fluidvolume per drop and measure the number of drops per unit of time. Whilethe illustrative example of FIG. 17 includes the first IV sensor and thesecond IV sensor, in some examples the first IV sensor and the second IVsensor can be the same sensor or same one or more sensors. Operation1708 can include to determine if a fluid drop was recognized by thesecond IV sensor. If in operation 1708 it is determined that a fluiddrop was recognized, operation 1710 can include determining a fluid flowrate, such as by calculating the flow rate by multiplying the number ofdrops per unit time by the volume per drop. In some examples, the volumeper drop is measured, while in other examples the volume per drop may beinput by a user, or can be a value retrieved from a memory. Operation1712 can include transmitting instructions to a display to cause a fluidflow rate to be displayed. Operation 1714 can include saving flow rateinformation to the storage device to document the fluid flow rate in theEMR. Any time a change is input or detected in the system, updated flowrate information can be displayed and saved.

If in operation 1708 it is determined that a fluid drop was notrecognized, operation 1616 can include receiving float information fromthe second IV sensor or another IV sensor. The float information caninclude information about a float in the drip chamber including is thefloat still (e.g., not moving), moving, or is movement of the floatslowing down. Operation 1718 can include determining if the float ismoving. If the float is moving, Operation 1720 can include determiningthe fluid flow is constant. In such a scenario, the fluid is flowing butthe fluid is flowing so quickly that individual drops of fluid cannot bedistinguished because the fluid is flowing as a steady stream. Operation1720 can further include determining the fluid flow rate by receiving IVbag physical characteristic information from a physical characteristicsensor, such as a weight sensor. The physical characteristic informationcan include weight information from the weight sensor (e.g., scale).Operation 1722 can include determining the fluid flow rate bycalculating the change in IV bag weight over a period of time. In otherexamples, instead of weight information, the physical characteristicinformation can include a position of the IV bag that changes as aresult of a change in weight, without the physical characteristic datacorresponding directly to a weight measurement. Other physicalcharacteristics and other physical characteristic sensors configured tomonitor IV fluid delivery may be provided such that an automated, or atleast partially automated EMR system is enabled.

If in operation 1718 it is determined that the float is not moving,operation 1728 can include determining that no fluid is flowing from theIV bag and transmitting one or more of: an instruction an actuator suchas a clamp, to cause the actuator to inhibit fluid flow to the patient(e.g., close the clamp onto IV tubing to prevent flow); and transmit andinstruction to an indicator (e.g., display, audible, tactile indicator)to cause an alert to be generated. Operation 1730 can include saving ano fluid event to the storage device.

FIG. 18 is a flow chart illustrating an example technique 1800 ofmedication identification and measurement. In operation 1802 a providerinserts a medication syringe into an injection portal (e.g., 411, FIG.4). In operation 1804 the medication can be identified by a sensor suchas by at least one of the RFID, barcode or QR sensors described herein.In operation 1806 processing circuitry checks for medication errors bycomparing the medication against one or more of: a doctor's orders,allergy history, medical history, other medications and current vitalsigns. In operation 1808, the results of the medication error check canbe displayed on an electronic record display. The results can indicateno error, the presence of an error, specific details about the error, orpresent a link to access information including additional details aboutthe error. In operation 1810, if a serious medication error isrecognized, the error deploys (e.g., causes actuation of) IV tubingclamps (e.g., 1060A, 1060B of FIG. 10) to prevent injection of themedication.

If in operation 1812, such as when no errors are determined, the machinevision camera and software can measure the diameter of the syringe. Inoperation 1814, an image of, or representation of the image of thesyringe, is displayed on the electronic record display. In operation1816 the provider squeezes the plunger of the syringe. In operation1818, the machine vision camera and software measure the distancetraveled by the syringe's plunger seal (e.g., 548, FIG. 5). In operation1820 the processing circuitry calculates the volume injected bymultiplying the syringe diameter times the distance of plunger travel.The processing circuitry can also calculate the dose by multiplying thevolume injected by the concentration of the medication. In operation1822 the injected dose and volume are displayed on the electronic recorddisplay. In operation 1824 the injected dose and volume are time stampedand recorded in the electronic medical record.

FIG. 19 is a second flow chart illustrating a technique 1900 includingaspects of the technique 1800 of medication identification andmeasurement from the perspective of processing circuitry (e.g., 157,FIG. 1; 257, FIG. 2; 1502, FIG. 15).

Technique 1900 can include an operation 1902 to receive medicationidentification information such as medication type, concentration,brand, lot number or amount, from a first medication sensor (e.g., RFID,barcode or QR reader). Operation 1904 can include saving medicationidentification information to a storage device (e.g., one or morestorage devices, memory). Operation 1906 can include comparingmedication identification information to at least one of a medicationorder, allergy history, medical history, other medications ordered forthe patient, and vital signs (e.g., previously obtained vital signs orcurrent vital signs of the patient via continuous monitoring). Operation1908 can include determining if a medication error is present. Operation1910 can include receiving syringe information from a second medicationsensor (e.g., a sensor configured to measure diameter, such as a machinevision camera). Operation 1912A can include receiving medicationdelivery information from the second medication sensor or anothermedication sensor. In some examples, the medication delivery informationcan include a distance of a syringe plunger travel.

Operation 1912B can include transmitting instructions to a display tocause an image of the syringe (e.g., actual image or representation ofthe syringe) to be displayed. A representation of the syringe caninclude an image communicating information about the syringe that is notan image of the actual syringe or can be a modified image of thesyringe, such as to highlight or point out aspects of the syringe ormedication within the syringe

Using the medication delivery information obtained in operation 1912A,operation 1914 can include determining a medication delivery amount.Operation 1916 can include transmitting instructions to a display (e.g.,display 226, FIG. 2) to cause the medication delivery information ormedication delivery amount to be displayed.

If in operation 1908 it is determined that a medication error ispresent, operation 1920 can include transmitting instructions includingerror information to the display or another display to cause the errorinformation to be displayed. In some examples, any of the instructionsdescribed herein that are sent to the display can be sent to one or moredisplays. Such displays can be located locally or remotely (e.g., in adifferent part of a room, in a separate room, in another building, inanother state, in another country), to alert multiple providers. Forexample, a provider such as a nurse anesthetist located adjacent to thepatient can be alerted to and provided with the information via display226. In addition, a second provider, such as an anesthesiologistsupporting the nurse anesthetist, and who may be supporting other nurseanesthetists working in different rooms, can also be alerted on adisplay of a mobile device, which may prompt them to check in with andpotentially assist the nurse anesthetist. This concept can be appliedoutside the operating room to manage medication delivered by providersworking in different rooms of a hospital or other care center, while asecond provider such as a nurse manager, nurse practitioner, pharmacistor doctor oversees the work of the first provider. In operation 1922 theerror information can be saved to one or more storage devices (e.g.,259, FIG. 2; 1516, FIG. 15).

Also in response to determining that a medication error has occurred inoperation 1908, operation 1924 can include transmitting instructions toan actuator such as an IV tubing clamp to inhibit (e.g., prevent,reduce, limit) injection. In some examples, the actuator can reduce orlimit the amount of the injection to a specified amount rather thancompletely inhibiting or preventing administration of the medication.Operation 1926 can include saving an inhibit injection event informationto a storage device, such as any of the one or more storage devices(e.g., 259, FIG. 2; 1516, FIG. 15). The inhibit injection eventinformation can include information such as the time of the event andthe action taken to inhibit injection and how much the injection wasinhibited (e.g., partially inhibited, completely inhibited, or amount ofmedication inhibited from injection).

If in operation 1908 it is determined that a medication error has notoccurred, operation 1910 can include receiving syringe informationincluding a syringe diameter from a sensor such as a machine visioncamera. In some examples, the sensor can be the first medication sensoror can be a second medication sensor. Operation 1912A can includereceiving medication delivery information from a sensor such as from thefirst medication sensor, the second medication sensor or another sensor.The medication delivery information can include a distance of plungertravel relative to a syringe body. Operation 1912B can includetransmitting instructions to one or more displays such as, display 226,FIG. 2, to cause an image of the syringe or representation of thesyringe to be displayed.

Operation 1914 can include determining a medication delivery amount,such as a volume or dose injected. For example, the volume injected canbe calculated by multiplying the syringe diameter by the distance ofplunger travel. The dose injected can be calculated by multiplying thevolume injected by the concentration of the medication.

Operation 1916 can include transmitting instructions to the one or moredisplays to cause the medication delivery information to be displayed.Operation 1918 can include saving medication delivery information to thestorage device (e.g., EMR). In some examples, the medication deliveryinformation can include, but is not limited to, volume, dose, time ofthe injection, or time period of the injection.

FIG. 20 is a flow chart illustrating an example of a second technique ofIV fluid identification and measurement including safety and securitymeasures. Aspects of technique 2000 can be similar or the same astechniques 1800 and 1900, however, technique 2000 is particularlywell-suited to the challenges of maintaining safety and security withcontrolled drugs such as narcotics. Technique 2000 can include operation2002 of identifying a medication (e.g., a controlled drug) andidentifying a health care provider, such as by RFID, barcode or QR codereader, retinal scanner, facial recognition, or fingerprint. Operation2002 can occur at the time a provider checks out a drug from a pharmacyor a medication dispensing machine. The medication can include anarcotic in a tamper-proof, non-refillable syringe.

Operation 2004 can include identifying a provider such as by RFID,barcode or QR code reader, retinal scanner, facial recognition, orfingerprint at a patient's bedside, such as at an injection portal(e.g., 411, FIG. 4). The provider can be the same or a differentprovider as the provider in operation 2002. In operation 2006, theprovider inserts the medication syringe into the patient's injectionportal. In operation 2008, the controlled drug is identified, such as byan RFID, barcode or QR code reader associated with the injection portal.Operation 2010 can include processing circuitry checking for medicationerrors by comparing the medication against doctor's orders, allergyhistory, medical history, other medications and vital signs. Operation2012 can include displaying medication error check results on a display,such as display 226, FIG. 2. If the medication error is of a seriousnature, the error can cause IV tubing clamps to prevent injection.Operation 2016 can include machine vision camera and software measuringthe diameter of the syringe. Operation 2018 can include an image, or animage representing the syringe being displayed on a display, such asdisplay 226, FIG. 2. Operation 2020 can include a provider squeezing theplunger of the syringe. Operation 2022 can include the machine visioncamera and software measuring the distance traveled by the syringe'splunger seal (e.g., 548, FIG. 5). Operation 2024 can include processingcircuitry determining the volume of medication injected by multiplyingthe syringe diameter by the distance of plunger travel or determiningthe dose of medication injected by multiplying the volume of medicationinjected by the concentration of the medication. Operation 2026 caninclude displaying the injected volume or dose on a display, such asdisplay 226, FIG. 2.

Operation 2028 can include saving the injected volume or dose along witha timestamp to the EMR. Operation 2030 includes repeating the operationsof technique 2000 as necessary until the machine vision camera andsoftware documents an empty syringe. Operation 2032 includes completingthe “chain of custody” for a specific syringe of controlled medication.The operations of technique 2000 can be repeated as necessary for othersyringes, thereby completing the “chain of custody” for each syringe.

FIG. 21 is a second flow chart illustrating a technique 2100 includingaspects of the technique 2000 of IV fluid identification and measurementincluding safety and security measures from the perspective ofprocessing circuitry, such as, but not limited to, processing circuitry157, FIG. 1; 257, FIG. 2; 1502, FIG. 15. The technique may involveprocessing circuitry 2202B, such as may be part of a medication vendingsystem as shown in FIG. 22. FIG. 22 illustrates generally an example ofa block diagram of vending system and a medication delivery system ofFIGS. 1-21 and 23 upon which any one or more of the techniques (e.g.,methodologies) discussed herein may perform in accordance with someembodiments. FIGS. 21 and 22 are described together.

In some examples, operations 2102 and 2104 can be part of a vendingsystem (2202, FIG. 22) for managing medication withdrawal from apharmacy or other vending system. Operations 2110-2136 can be part of amedication delivery system (e.g., can be used with the bedside patientsystems and modules shown and describe in FIGS. 1-15; medicationdelivery system 2210, FIG. 22). Operation 2138 can tie information,including data generated by the vending system 2202 and the medicationdelivery system 2210 together to facilitate tracking a “chain ofcustody” for a specific syringe of controlled medication from thepharmacy until the medication is completely injected into the patient.Chain of custody information can be stored to one or more of: thevending system storage 2202A, the medication delivery storage 2216, andchain of custody storage device (e.g., 2206, FIG. 22) and the EMR. Anyof the storage described herein can include one or more storage devicesor memory as described herein and can include other storage devices inelectrical communication with the vending system or the medicationdelivery system.

Operation 2102 of the vending system can include receiving withdrawingprovider identification information from a medication dispensing sensor2202C, such as a first RFID or barcode reader that reads a badge of aprovider and reads the medication identification information from asyringe or other medication container, or any other type of suitablesensor described herein. Operation 2104 can include associating andsaving the medication identification information and the withdrawingprovider identification information to a vending system storage device(e.g., 2202A, FIG. 22).

Operation 2110 can include receiving medication identificationinformation from a first identification sensor (e.g., RFID, QR, barcodereader, or machine vision camera reads information about a medication)and receiving delivery provider identification information from thefirst identification sensor or another identification sensor (e.g., asecond identification sensor, another RFID, QR or barcode reader,machine vision camera, retinal scanner, facial recognition sensor orfingerprint reader). In some examples, patient identificationinformation can also be obtained from one of the first identificationsensor, second identification sensor or another identification sensor,such as by scanning patient identification information on a hospitalwristband. In some examples receiving the medication identification, theprovider identification information or the patient identification cancause the processing circuitry to send an instruction to a display toprompt the user for the other of the medication identificationinformation, the provider identification information or the patientidentification information.

Operation 2112 can include comparing the received identificationinformation to one or more of: a medication order, allergy history,medical history, other medications and current vital signs. Operation2114 can include determining if a medication error is present. If it isdetermined that a medication error is present, operation 2116 caninclude transmitting instructions including error information to adisplay to cause the error information to be displayed. Operation 2118can include saving the error information to one or more storage devices.Further, if in operation 2114 it is determined that a medication errorhas occurred, operation 2120 can include transmitting inhibit injectioninstructions to an actuator such as , but not limited to, an IV tubingclamp (e.g., 1060A, 1060B; FIG. 10) to inhibit injection. Operation 2122can include saving an inhibit injection event information to one or morestorage devices.

Operation 2124 can include receiving syringe information from a secondmedication sensor (e.g., syringe diameter including syringe innerdiameter, outer diameter, or wall thickness from a machine visioncamera). Operation 2124 can include receiving syringe size informationfrom a data storage device, the syringe size information provided by thesyringe manufacturer that supplies the specific syringes used by thespecific healthcare facility. Operation 2126 can include transmittinginstructions to a display to cause an image of the syringe or arepresentation of the syringe to be displayed. Operation 2128 caninclude receiving syringe movement information from the secondmedication sensor or another sensor. Syringe movement information caninclude, for example, a distance of travel of the syringe plungerrelative to the syringe barrel.

Operation 2130 can include determining medication delivery informationbased on the syringe movement information. Medication deliveryinformation can include, for example, a volume or dose of medicationdelivered to the patient (e.g., ejected from the syringe). In someexamples, the volume of medication delivered (e.g., ejected from thesyringe) can be calculated by multiplying the syringe inner diameter bythe distance of plunger travel. Likewise, the dose of medicationdelivered can be calculated by multiplying the calculated volume by aconcentration of the medication. Operation 2132 can include savingmedication delivery information to one or more storage devices. In otherwords, operation 2132 can include documenting volume, dose and time inan EMR, in some cases automatically without intervention from aprovider.

Operation 2134 can include transmitting instructions to one or moredisplays described herein to cause the medication delivery informationto be displayed. Operation 2136 can include determining that a syringeis empty and saving “chain of custody” complete for the specific syringeof medication (e.g., controlled drug) to one or more storage devices.

To complete and document the chain of custody, thereby ensuring themedication was delivered to the patient, operation 2138 can include oneor more of receiving, associating and saving to one or more chain ofcustody storage devices (e.g., 2206, FIG. 22), information from both thepharmacy vending system 2202 (FIG. 22) and the bedside medicationdelivery system 2210 (FIG. 22)(e.g., 1516; FIG. 15). In some examplesthe one or more chain of custody storage devices is not necessarilyseparate from the vending system storage 2202A or the medicationdelivery storage 2216, but rather can reside with one or the othersystems, a different system, multiple systems or can be included as asingle storage device.

FIG. 23 illustrates and example technique 2300 for assessingphysiological events. In some examples, the EMRs created by theautomated data consolidation module 100, 200, 400, 600, 800 can providethe most accurate and temporally correlated information about therelationship between any injected medication and the resultingphysiologic response. In some examples, this is uniquely accuratedose-response data can be used as a final check of the chain of custodyfor controlled medications or any medication. The processing circuitry157, 257 may include or be in electrical communication with artificialintelligence (AI) and/or machine learning that can compare the measuredphysiologic response in the several minutes after a medication isinjected, to the expected physiologic response for that dose of thatmedication. For example, if the injected medication was a narcotic, itwould be expected that the heart rate and blood pressure of the patientwould decrease quickly after the injection.

In some examples, if the expected physiologic response does not occur,the AI software of the automated data consolidation module 100, 200,400, 600, 800 may electronically “flag” that injection as suspicious.For example, if there is no physiologic response after injecting whatwas supposed to be a narcotic, it is possible that the drug had beenstolen and replaced by saline. On the other hand, no response may simplymean that the patient is addicted to and tolerant of narcotics and thattoo is worth noting. An unpredicted response does not prove anything butmultiple unpredicted responses in multiple patients can be suspicious.Therefore, aggregating or analyzing data over time for a particularpatient or provider can alert management to issues. If any individualprovider traverses a threshold number of flags (e.g., too many “flags”)for unexpected physiologic responses (including no response), theautomated data consolidation module 100, 200, 400, 600, 800 can generatean alert to notify management and an investigation of that provider maybe warranted. Knowing that AI is “watching” the patients' response to ahealthcare providers' injected medications, can be a significantdeterrent to tempted drug thieves.

Technique 2300 can include determining if one or more unexpectedphysiological events has occurred, analyzing saving, aggregating anddisplaying such information, in any order. The method can be performedby processing circuitry 157, 257, 1502, including other processingcircuitry, memories and databases in electrical communication withprocessing circuitry 157, 257, 1502 to one or more of: receivephysiologic data, analyze physiologic data, determine physiologic datais unexpected, create and send instructions to cause an alert to theprovider or another user, or save a physiological event information to astorage device 1516 which may include a database. The physiologicalevent information can include, but not limited to data generated by thevarious sensors and equipment described herein, including one more of:physiological information, patient information, provider information,medication information, time information, location information, facilityinformation, equipment information, aggregated physiological eventinformation and analyzed physiologic event information.

Operation 2302 can include receiving physiologic data from a physiologicsensor, Operation 2304 can include analyzing the physiologic data.Operation 2304 can include comparing the physiologic data to expectedphysiologic responses. Based on the outcome of the analysis in operation2304, in operation 2306, the processing circuitry can determine if thephysiologic data is unexpected, and if so, operation 2308 can includesaving unexpected physiologic event information to one or more storagedevices, or can include sending instructions to one or more displays todisplay unexpected physiologic event information.

Operation 2310 can include aggregating or analyzing physiologic eventinformation from a plurality of unexpected physiological events andgenerating aggregated or analyzed physiologic event information. In someexamples, aggregating can include aggregating a number of physiologicalevents by counting the number of physiological events for a givenprovider, patient, group of patients, medical facility, type ofmedication, or any other suitable assessment. Operation 2312 can includesaving aggregated or analyzed physiologic event information to one ormore storage devices or sending instructions to one or more displays todisplay aggregated or analyzed physiologic event information. In someexamples, the physiologic event information can include any type ofphysiologic event that occurs, including expected or desirablephysiologic events

The operations of technique 2300 can help provide safer care forpatients, including providing narcotic medications when helpful, whilekeeping a close eye on drug abuse by providers or patients. Taken at ahigh level, technique 2300 can help medical facilities evaluate whichmedications are most often abused by patients or stolen by providers,and to mitigate risk for insurers.

Errors can easily occur during the administration of oral medications.Administering oral medications and monitoring the administration of oralmedications has always been challenging. Many pills look alike and areeasy to confuse with similar pills. Even dissimilar pills are easilyconfused. Pills are easily miscounted. Finally, the patient may not takethe pill and swallow it. Automating the documentation of oral medicationadministration may have several advantages including: reducingmedication errors, alerting the caregiver to a medication error orallergy, verifying the pill count and verifying that the patientactually takes and swallows the pills.

In some examples, the pills may be delivered from the pharmacy to theward, to long-term care, to home care or any other care location, in abottle labeled with an RFID tag or barcode label. The RFID tag orbarcode label may identify one or more of the patient, the medication,dosage, manufacturer, expiration date and other useful information.

In some examples, the monitoring of the administration of oralmedications may be accomplished by a combination of RFID, barcode, videoand machine vision combined with machine learning (ML) technologies. Forexample, a bottle of pills may be identified by a RFID tag or barcodelabel that can be read by a RFID or barcode reader that is in electricalcommunication with the processing circuitry and software of the module.In some examples, the processing circuitry and software of the moduleautomatically checks the identified pill bottle against the doctor'sorders to verify that they match.

In some examples as shown in FIG. 27, the caregiver or a mechanicaldispenser pours the prescribed number of pills 2702A-D from the bottleinto a container or onto a small tray 2704 of a pill monitoring system2700. In some examples, the pills 2702A-D are then “viewed” by a digitalcamera 2706 such as a video camera or machine vision camera and theimage is sent to the processing circuitry and software of the modulewhere it is compared with known images of the specific pills stored inan image library, using ML. The identity and quantity of the pills2702A-D may be compared to the doctor's orders and the identity of thepills 2702A-D may be compared to the bottle of pills that was identifiedby a RFID tag or barcode label. In some examples, if RFID or barcodesare not used to identify the pill bottle, it may be advantageous for themachine vision image of the pill 2702A-D itself to be compared to knownimages of the pills prescribed by the doctor stored in an image library.In some examples, machine vision may be used to measure the size of thepill 2702A-D in order or help differentiate similar colored or shapedpills. In any event, verification of the number and identity of thepills to be given to the patient at that time, can be made andautomatically recorded in the electronic record.

In some examples, the caregiver or patient manually indicates to theprocessing circuitry and software of the module, that the pills 2702A-Dwere taken by the patient. In some examples, the processing circuitryand software can then record the pills 2702A-D taken as a “dose event”including a time-stamp that allows temporal correlation with thepatient's response to the medication.

In some examples, a video camera provides images of the patient takingthe pills 2702A-D to the processing circuitry and software of themodule. The processing circuitry and software may use ML to compare thevideo images to known images of patients taking pills stored in an imagelibrary, in order to automatically confirm that the pills were taken bythe patient. In some examples, the processing circuitry and softwarethen record the pills 2702A-D taken as a “dose event” including atime-stamp that allows temporal correlation with the patient's responseto the medication.

In some examples, the video camera images may be used to document thatthe patient took their pills 2702A-D even in the absence of a healthcareprovider being present, such as a patient at home. A surprisingly highpercentage of patients are unreliable in taking their medications. Thismedication documentation system not only documents that the medicationswere taken but may also be programed to provide positive feedback to thepatient to reinforce the good behavior. Alternatively, if themedications are not being taken as ordered by the healthcare provider,the medication documentation system may notify the healthcare providerso that they can intervene sooner.

In some examples, the module of this disclosure includes a system fordocumenting healthcare events and medical equipment operating settings,using machine vision or video with machine learning (ML) or artificialintelligence (AI) analysis. In some examples as shown in FIG. 28, aportable digital camera 2800 may be used to document the event orequipment settings. Digital cameras 2800 in this disclosure include butare not limited to: RGB digital cameras, black and white digitalcameras, infrared digital cameras, video digital cameras and machinevision digital cameras. In some examples, the portable digital camera2800 may be a handheld digital camera and may be conveniently mounted ona pistol-like handle 2802 and include a pistol-like trigger 2804 foractivating the camera 2800. The pistol-grip handle may include adisposable cover to prevent contamination of the handle by the user'scontaminated hands. In some examples, the digital camera 2800 producesdigital images that may be sent to the processing circuitry in themodule, where machine learning (ML) or artificial intelligence (AI)software can compare the images with known images stored in an imagelibrary. In some examples, the digital camera 2800 produces digitalimages that may be sent to the processing circuitry in the module, wheremachine vision software may measure the size of an object, the locationof an object or the distance an object moves. In some examples, thedigital camera 2800 produces digital images that may be sent to theprocessing circuitry in the module, where machine vision software maymeasure subtle changes in color, for example changes in skin color seenduring changes in perfusion, oxygenation or remote plethysmography.

In some examples, the handheld digital camera may include a laserpointer 2806 that helps the user accurately aim the camera tostandardize the scene and to most effectively capture an image of theitem or device. The laser pointer 2806 may also be used to identify thesubject item or device in a scene of multiple items or devices, to theprocessing circuitry and ML software. In some examples, a label with aprinted target may be the designated target to aim the laser pointer2806 at, standardize the scene so that the resulting digital photographor image is oriented similarly to the comparative images in the ML imagelibrary. In one example, the preferred camera orientation may bestraight at the target and approximately 2 feet away from the item ordevice. Other icons or shapes are anticipated for the target labels.

In some examples as shown in FIGS. 29 and 30, the item or device 2900being photographed may be an anesthetic vaporizer and may have a labelsuch as a target 2902 attached so that the laser pointer 2806 can beaimed at a specific location. Aiming the laser 2806 at a specificlocation on the item or device 2900 helps to standardize the scene sothat images of the item or device 2900 are always taken from the sameorientation—the straight-on view of the subject item or device forexample. This eliminates the need for the image library to includeimages of the sideview, the back view, the top view or oblique angles,which would vastly complicate the ML identification of the item ordevice 2900. Consistently photographing the subject item or device 2900from a consistent reference point such as straight on to the target2902, standardizes the scene allowing the image library to be muchsmaller, eliminating the need for side view and top view images.

In some examples, if more than one item or device are in thephotographed image, the laser pointer 2806 aimed at target 2902standardizes the scene by identifying for the processing circuitry, theitem or device that is to be documented.

In some examples, the laser pointer 2806 may be a diode laser or anyother kind of laser. In some examples, the laser pointer may be a diodelight or any other kind of light. In some examples, the laser pointermay be one color (red for example) while aiming the camera lens 2808 andswitch to another color (green for example) when the digital photographhas been successfully taken. The laser pointer 2806 not only improvesthe quality of the captured image but makes the job of capturing theimage more fun for the clinician. For example, in some examples, theindication of a successfully captured image may be a projected imagesuch as the image of a paintball splattering against the target 2902 onthe item or device 2900. Other projected images including but notlimited to: arrows, darts, splattering cream pies, splattering eggs,splattering tomatoes or even bullet holes, are anticipated for makingthe capture of data more fun. Aiming the laser 2806 at a specific target2902 on the item or device 2900 also helps to steady the camera 2800before shooting the picture.

In some examples, the projected image of a splattering paintball forexample may be from a portable liquid crystal display (LCD) projector2810 that may use a light emitting diode (LED) as a light source,mounted in the pistol-like handheld digital camera holder 2800. Othertypes of projectors or even clusters of lasers are anticipated asalternative ways to project an image onto the item or device. In someexamples, when the image of the item or device is successfully recordedby the digital camera, the LCD projector 2810 may instantly project ashort video sequence of a paintball (or other object) splatteringagainst the target 2902 on the item or device 2900 to give positivefeedback to the clinician of a successful image acquisition.

In some examples, the system may also keep score of the number of“hits,” the percentage of “hits,” or the accuracy of the “hits,” inorder to make a game of data recording. Having fun recording healthcaredata is the exact opposite of clinician's current attitude toward datarecording. Manual data recording as it is currently done in healthcareis described as “hated work” and has been shown to cause “burnout,” evenleading to early retirement for some clinicians. In some examples, thepoint and shoot nature of the portable digital camera 2800 of thisdisclosure with instant positive feedback for a “hit,” is designed tomake data entry fun for clinicians.

In some examples as shown in FIG. 29, the item or device 2900 to bedocumented may include a label 2906 with a QR code 2904 attached to theitem or device 2900. In this disclosure we refer to a QR code 2904,however it should be understood that other coding technologies includingbut not limited to: barcodes, symbols, icons or alpha numericidentification, are anticipated as acceptable replacements for QR codes2904. In some examples, the label 2906 with a QR code 2904 may be thetarget or may be part of the target 2902 and is attached to the item ordevice 2900 in the scene to be photographed with the portable digitalcamera 2800. In some examples, the QR code 2904 may be inside the target2902 previously disclosed and may be applied to a substantially centrallocation on the item or device 2900 in order to standardize the scene.In some examples, the target 2902 with the QR code 2904 label 2906 maybe applied to an area of the item or device 2900 that aims the camera2800 so that the area of interest 2910 of the item or device 2900, suchas a dial 2908 or indicator light for example, is captured in the image.In some examples, the area of interest 2910 for each item or device inthe image library may be preidentified by the processing circuitry sothat the ML software know where in the image to look for the keyinformation.

In some examples, it may be preferrable to print the target 2902 and QRcode 2904 onto a non-reflective material so that the laser beam is notreflected back at the user or other staff. In some examples, the laserpointer 2806 may automatically turn off just before the digital image istaken so that the laser 2806 does not distract from reading the QR code2904.

In some examples, the QR code 2904 can be read by the handheld digitalcamera 2800 and processing circuitry to identify the type of item,device or equipment, the manufacturer and model of the item, device orequipment as well as other information that may be useful. For example,as shown in FIG. 29, the QR code 2904 may indicate that the device is anIsoflurane anesthetic vaporizer made by Drager, Model XXX. Byidentifying the specific device with a QR code 2904, the ML softwaredoes not need to look through the entire image library to first identifythe type of the item or device, then to identify the make and model ofthe item or device and then finally identify the desired informationfrom the item or device.

In some examples as shown in FIGS. 30A-D, the image of an anestheticvaporizer 3000A taken by handheld digital camera 2800 is compared toimages of similar anesthetic vaporizers 3000B-D stored in the imagelibrary. The QR code 3004A allows the software to instantly narrow thesearch to images of that specific device with different dial settings3002B-D in order to determine the concentration of the Isoflurane comingfrom the vaporizer. In some examples, the scene may be furtherstandardized by preprograming into the processing circuitry memory thepart of the image containing the important information or area ofinterest for that specific item or device. In some examples, theprocessing circuitry may draw an imaginary box around the area ofinterest 3008A-D in the scene. In this example, the dial 3002 is theimportant portion of the scene and the machine learning comparison ofimages with images in the image library can focus on the part of theimage in the box 3008A-D, the area of interest. In this example, thedial setting 3002A in the subject image as shown in FIG. 30A, matchesbest with the dial setting 3002C in the image in FIG. 30C and the MLsoftware has been “taught” that the image in FIG. 30C corresponds with2% isoflurane. When the image is successfully matched, “2% Isoflurane”is then automatically time-stamped by the processing circuitry and savedto the anesthetic record, the electronic medical record or otherdatabase as a dose event.

In some examples, with the make and model of the vaporizer 3000Aidentified by the QR code 3004A, the ML software only needs to review˜10 images while looking through anesthetic vaporizer images 3000B-D inthe image library, in order to find a match that indicates that the dialis set at 2%. Tens of thousands of images would be necessary if thedevice 3000A was not identified by a QR code 3004A and the orientationof the device in the scene not prescribed by the laser pointer 2806 andtarget 3006A. The QR code 3004A read by the handheld digital camera 2800identifying the specific device 3000A vastly narrows the ML imagesearch, making the search for a match both faster and more reliable.

In some examples, the handheld digital camera 2800 with machine visioncan be used to document the operating parameters of a piece ofelectronic or electromechanical equipment that does not produce a dataoutput of its operating parameters. For example, the handheld digitalcamera 2800 with machine vision may be used to capture an image of thecontrol panel of an electrosurgical unit (ESU) 3100 as shown in FIG. 31.In this example, a QR code 3104 may be used both as an aiming target3106 and to identify the make and model of the specific piece of medicalequipment. In some examples, the processing circuitry may draw animaginary box around the areas of interest 3112AB in the scene. In thisexample, the germane information is the wattage of current in the cut3108 and coagulation 3110 modes of operation, indicated by illuminatednumbers on the control panel. The machine learning comparison of imageswith images in the image library can focus on the part of the image inthe areas of interest 3112A,B boxes. In this example, the ML softwareneeds to match these numbers 3108, 3110 with numbers in its imagelibrary to determine the power settings and then those values may betime-stamped by the processing circuitry and saved to the anestheticrecord, the electronic medical record or other database as a dose event.

In some examples, “standardizing the scene” using a light pointer andcoding system results in one or more of the following benefits: 1.)Steadies the camera. 2.) Identifies which item or device is the subjectwhen more than one item or device may be in the image. 3.) Makes thefront view orientation of the camera relative to the subject item ordevice reproducible, eliminating the need for side view, back view, topview or oblique view images of the item or device in the image library.4.) Specifically identifies the type of item or device, make and modelso that only images of that make and model are used for comparison andmatching, vastly narrowing the search. 5.) Allows preprogramming of theprocessing circuitry to know where in the image the importantinformation is located so that the machine learning function can focuson that area—the area of interest. In some examples, “standardizing thescene” as disclosed herein results in ML image matching that is muchmore reliable and faster.

Since the control of the ESU is typically done by the OR nurse, it maybe advantageous to have two or more portable digital cameras 2800 withmachine vision in the operating room. One camera may be used primarilyby the anesthesia provider and the other camera by the OR nurse. Thecameras may be in wired or wireless electronic communication with theprocessing circuitry and software of the automated data entry system.

In some examples, the handheld digital camera 2800 with machine visioncan be used to document other aspects of the patient's care. Forexample, if it is desirable to document the placement of an IV in thepatient's left hand, the handheld digital camera 2800 may be aimed atthe IV in the patient's hand and an image acquired. In this example, theML software first must identify that it is an IV and then identify aleft hand. Since IV hubs are color coded corresponding with the size ofthe IV catheter, the ML software can also identify the size of the IVcatheter.

In some examples, a short procedure note may be desirable or required.Since the processing circuitry and software of the system fordocumenting healthcare events is specifically designed to aid thehealthcare provider, it needs to know who the provider is to optimizethe assistance. Therefore, in some examples the provider may start theirinteraction with the system by identifying themselves with the RFID orQR code or barcode on their ID badge. Since healthcare providers tend tobe “creatures of habit,” they almost always use the exact samedescription of a given procedure when documenting that procedure, everytime they do that procedure. If provider's preferred description of anygiven procedure has been “taught” to the processing circuitry andsoftware and stored in the memory of the system, the system fordocumenting healthcare events of this disclosure can automatically enterthat custom description of the procedure into the record and furtherautomatically customize the description with “left hand” and “18 gauge”in this example.

In some examples, the handheld digital camera with machine vision can beused to document other aspects of the patient's care. For example, if itis desirable to document intubation of the trachea with an endotrachealtube, the handheld digital camera 2800 may be aimed at the patients facewith a tube protruding from the mouth and an image acquired. In thisexample, the ML software first must identify that it is an endotrachealtube. In some examples, the size of the tube may be obtained from a QRcode or barcode on the waste sterile wrap that may be placed in thefield of the image. Alternately, the tube size and depth may be assumedbased on the known preferences of the given clinician. For example, agiven clinician may routinely intubate adult males with 8mm endotrachealtubes taped at 21cm at the teeth. The processing circuitry and softwareknow that the patient is an adult male and automatically fills in theprocedure note for the clinician. If there is an unlikely exception tothe routine, the clinician can go into the record and manually correctthe record for the exception.

Since healthcare providers tend to be “creatures of habit,” they almostalways use the exact same description of a given procedure whendocumenting that procedure, or other healthcare notes. In some examples,common procedure notes or common healthcare notes may be prerecorded bya clinician and the prerecorded note may be labeled with a QR code orbarcode. In some examples, a list of these prerecorded notes may beconsolidated on one or more cards or sheets of paper along with theircorresponding QR code or barcode. In some examples, the handheld digitalcamera 2800 with machine vision can be used to scan the appropriate QRcode or barcode which automatically causes the processing circuitry torecord the chosen procedure note or healthcare note into the anestheticrecord or EHR, saving the clinician considerable time.

In some examples, the handheld digital camera 2800 with machine visioncan be used to document subtle color changes in the patient's skin. Inthis example, consistent illumination of the skin may be important andthe handheld digital camera 2800 may include one or more illuminatinglights shining at the scene to be photographed. The lights may be white,colored or of specific wavelengths, in order to highlight the desiredobservable parameter. For example, if remote plethysmography is theobjective, illumination in the infrared and near-infrared spectrum maybe desirable.

In some examples as shown in a top view in FIG. 32, if the distance ofthe handheld digital camera 3200 from the subject is important foraccurate image recognition or distance or movement measurements by theML or machine vision software, two laser pointers 3202AB may be used.The laser pointers 3202AB may be mounted on opposite sides of thehandheld camera 3200, separated from each other by one or more inches.The pointers are aimed inward to form a triangle with the two beams3204AB crossing at a prescribed point 3206 in front of the camera, forexample, two feet ahead of the camera. Therefore, when the handheldcamera 3200 is aimed at the QR code target, if only one point of laserlight 3206 is seen, the target is at the crossover distance which is thepreferred distance. As shown in FIGS. 33A-C, if two points of light3302ABDE are visible near the target 3304AC (FIGS. 33A and 33C), thecamera needs to be moved either toward or away from the target 3304ACuntil the two points of light merge into a single point of light 3302C(FIG. 33B) at the target 3304B. The two laser beams may be used totriangulate a preferred distance for the digital video or machine visionimage. Similarly, if machine vision is being used to determine the sizeof an object—a pill for example, the two laser beams may be used totriangulate a preferred distance for the digital camera from the objectbecause the distance of the digital camera from the object is criticalwhen measuring or calculating the object's size with machine vision.

While this disclosure has referred to handheld digital cameras 2800, itshould be noted that all portable digital cameras are anticipated. Forexample, a portable digital camera may be mounted on the users' glasses,or head band or cap or helmet or clothing. Any of these portable digitalcameras may deliver the images of this disclosure. In some examples, theportable digital camera may be temporarily fixed in position, mounted toan IV poll for example, in order to record a scene. The portable camerasmay be in wired or wireless communication with the processing circuitryof the module.

In some examples, response events may also be documented with thehandheld digital cameras 2800 of this disclosure. Including but notlimited to many vital signs monitors such as pulse oximeters andautomatic blood pressure machines, are free standing entities that donot produce a data output that can be recorded. Thus the data that theydisplay to the user must be manually recorded. In some examples, thehandheld digital cameras 2800 of this disclosure can be used toautomatically record the data displayed on the vital signs monitors orother physiologic measurement instruments.

In some examples, dose events that may be documented by the handhelddigital camera 2800, the processing circuitry and ML or machine visionsoftware of the module of this disclosure include but are not limitedto: oral medications, IV medications, patient positioning and movement,volatile anesthetics, ventilator settings, inspired gasses, airwaymanagement (ie. intubation, mask ventilation), pneumoperitoneuminsufflation pressures, patient warming parameters, surgical tableTrendelenburg angles, train-of-four stimulation, nursing activities,surgery, procedures such as IV canulation and feeding.

In some examples, response events that may be documented by the handhelddigital camera 2800, the processing circuitry and ML or machine visionsoftware of the module of this disclosure include but are not limitedto: legacy vital signs monitor reporting, remote photoplethysmography,skin color changes, airway pressures, urine output surgical blood loss,chest tube output and train-of-four response.

In some examples, equipment operating settings that may be documented bythe handheld digital camera 2800, the processing circuitry and ML ormachine vision software of the module of this disclosure include but arenot limited to: electrosurgical unit (ESU) settings, OR tablepositioning, pneumoperitoneum insufflation, sequential compressionleggings, smoke evacuator settings, UV disinfection documentation,Archimedes air mattress, surgical sponge count, maximum skin supportpressures and bed positioning.

As previously discussed in this disclosure, the current standard forpatient monitor display location in the anesthesia space in theoperating room is beside or behind the anesthesia provider. This is apoor design because a monitor alert (whether real or erroneous) focusesthe anesthesia providers attention away from the patient. In someexamples, it may be useful to use the design of an airplane cockpit asan example—the windshield and instruments are all in front of the pilot.During WWII it was realized that even looking down at the instrumentsvery briefly could be distracting, so the early “head-up displays” (HUD)were developed to move the instrument information up and into the viewout of the windshield. The origin of the name “head-up displays” stemsfrom a pilot being able to view information with the head positioned“up” and looking forward, instead of angled down looking at instrumentsbelow the windshield. In some examples, the HUD of this disclosure wouldproject data and waveforms from the physiologic monitors along withother important data and messaging to the space directly above thepatient's head, neck or chest for easy, convenient and distraction-freeviewing by the anesthesia provider.

Since WWI, HUDs have been developed for many airplanes and cars. HUDstypically consist of three components: a control module that producesthe image and determines how the image should be projected; a projectorunit which is typically an LED or LCD display/projector; and a combiner,which is the surface onto which the image is projected (generally acoated windshield glass or retractable glass screen above thedashboard). The transparent combiner is necessary in order to seethrough the projected information while viewing the scene ahead of theplane or car.

In some examples, it is advantageous to place the monitor displays asclose to the patient's head as possible so that the monitor and thepatient can be viewed in the same field-of-vision. In some examples asshown in FIG. 12, the module 1200 may be used to mount monitor display226 in a position directly above the patient's 202 head. In this case,module 1200 is advantageously located adjacent the patient 202 andtherefore is in an ideal location for mounting monitor display 226. Insome examples as shown in FIG. 13, the module 1300 may be used to mountmonitor display 326 in a position adjacent the patient's 302 head. Inthis case, module 1300 is advantageously located adjacent the patient202 and therefore is in an ideal location for mounting monitor display326.

Unlike the HUD in a plane or car, there is little need to see “through”the projected information while viewing the patient. Therefore, in someexamples it may be advantageous to project the information onto aprojection screen above the patient's head or even the verticalanesthesia screen portion of the surgical drape above the patient'shead. Unlike the HUD in a plane or car, mounting a rigid sheet of glassor plastic directly above the patient's head may encumber theanesthesiologist during procedures such as induction or emergence fromanesthesia, intubation, or even CPR. Therefore, in some examples it maybe advantageous to project the information onto a flexible projectionscreen or flexible sheet of plastic that can easily be moved out of theway if it is in the way of the anesthesiologist doing his or her work.

Unlike the HUD in a plane or car, where the projector is below thecombiner aiming upward, placing a projector directly in front of thepatient's face will inevitably be in the way. Therefore, in someexamples it may be advantageous to project the information onto aprojection screen or flexible sheet of plastic, from above or from thesides of the projection screen or flexible sheet of plastic.

In some examples as shown in FIGS. 34 and 35, a HUD 3404 and 3504 may besubstituted for the flat screen display 226 shown in FIG. 12. In someexamples as shown in FIGS. 34 and 35, HUDs 3404, 3504 may be mounted tomodules 3400, 3500 and positioned over the patient's head or neck orchest so that the combiners 3408, 3508 can be viewed in the same fieldof vision as the patient's head.

In some examples, the HUDs 3404, 3504 may include a support member 3406,3506 that can support a flexible combiner 3408, 3508 and maintain theflexible combiner 3408, 3508 in a predetermined shape such as flat orcurved. In order to prevent the combiner 3408, 3508 from injuring thepatient 3402, 3502 or preventing the anesthesiologist from caring forthe patient 3402, 3502, the combiner 3408, 3508 may be made of aflexible material including but not limited to plastic film, coatedplastic film, fabric, coated fabric, foam or combinations of thesematerials. The combiners 3408, 3508 may be made of clear or opaquematerials. Various coatings that have been developed for automotive HUDsor projection screens, may be utilized to better reflect the images,making them more visible. Rigid combiners 3408, 3508 made of plastic orglass are also anticipated. It is also anticipated that the anesthesiascreen can be used as a combiner.

In some examples, the support members 3406, 3506 are adjustably mountedto modules 3400, 3500 through a second support member 3412, 3512 and oneor more hinges. Support members 3406, 3506, 3412, 3512 of many sizes,shapes and adjustable constructions are anticipated.

In some examples, the HUDs 3404, 3504 may include one or more projectorunits 3410, 3510. In some examples, the projector units 3410, 3510 arelight emitting diode (LED) or liquid crystal display (LCD)displays/projectors. Other types of projectors are anticipated. In someexamples as shown in FIG. 34, the projector unit 3410 may be mountedabove the combiner 3408, shining downward on to the combiner 3408. Insome examples as shown in FIG. 35, the projector unit 3510 may bemounted at the side of the combiner 3508, shining laterally on to thecombiner 3508. It is also anticipated that the projector unit could bemounted next to the patient's head, shining upward on to the combiners3408, 3508 or could be mounted behind the combiners 3408, 3508, shiningat the back side of a transparent combiner 3408, 3508.

In some examples, the processing circuitry of modules 3400, 3500 providethe image data to the projector units 3410, 3510.

FIG. 36 is a flow chart illustrating a process 3600 for automaticallyrecording dose events using a portable digital camera 2800 andprocessing circuitry, such as, but not limited to, processing circuitry157, FIG. 1; 257, FIG. 2; 1502, FIG. 15. FIG. 36 illustrates generallyan example of a block diagram of process 3600 for automaticallyrecording dose events using a portable digital camera 2800.

In some examples, the process for automatically recording dose eventsstarts at 3602 when the healthcare provider desires to automaticallyrecord and document a healthcare dose event such as changing theconcentration of an anesthetic gas produced by the item or device,anesthetic vaporizer 2900, 3000 for example. In some examples, inoperation 3604, a handheld digital camera 2800 is equipped with a laserpointer 2806 and the laser pointer 2806 is aimed at a target 2902, 3006,3106 or icon attached to the item or device 2900, 3000 in a prescribedlocation. In some examples, the target 2902, 3006, 3106 is attached tothe front of the item/device 2900, 3000 to “standardize the scene” orstandardize the viewing angle of the camera relative to the item/device2900, 3000.

In some examples in operation 3606, when the laser pointer 2806 iscorrectly aimed at the target 2902, 3006, 3106 the trigger is pulled andthe handheld digital camera 2800 simultaneously records both a QR code2904, 3004, 3104 (or other coding technology) attached to the item ordevice 2900, 3000 near or within the target 2902, 3006, 3106 and animage of the item/device 2900, 3000.

In some examples in operation 3608, the processing circuitry processesthe data from the digital camera 2800, first reading the QR code 2904,3004, 3104 to identify the specific type of item/device 2900, 3000, 3100the make and the model. In some examples, the processing circuitry andmemory include information on the specific area of interest 2910, 3008,3112 for that item/device 2900, 3000, 3100. For example, the area ofinterest 2910 for vaporizer 2900 is the concentration dial 3002 at thetop of the vaporizer 2900. Since the metal can is of little interest,the ML software may be programed to focus on the area of interest 2910,ignoring the rest of the item/device 2900.

In some examples in operation 3610, the processing circuitry uses theinformation from the QR code 3004 to query an image library stored inthe memory. Rather than comparing the image of the subject item/device3000A to every image in the library, the query can be focused on onlyimages of items/devices 3000B-D identical to the subject item/device3000A and those images of identical items/devices 3000B-D may be storedtogether in a labeled file in the memory for easy access. This reducesthe inquiry/comparison of images by many orders of magnitude, bothspeeding the process and improving accuracy.

In some examples in operation 3612, the processing circuitry uses MLtechniques to compare the image of the subject item/device 3000A, toimages in the image library of items/devices 3000B-D identical to thesubject item/device 3000A. The different images of items/devices 3000B-Din the image library may be identical except for the control dial3002B-D being photographed at the various possible settings throughoutthe entire range. When the processing circuitry successfully matches theimage of the control dial 3002A of the subject item/device 3000A withthe image of the control dial 3002C of a known item/device 3000C in theimage library, the numerical setting on the control dial 3002A of thesubject item/device 3000A can assumed to equal the known numericalsetting of the known item/device 3000C in the image library.

In some examples in operation 3614, the processing circuitry takes thenumerical setting or numerical dose event and automatically adds atime-stamp to the dose event data. The processing circuitry then reportsthe time-stamped dose event data from the subject item/device 3000A toat least one of the EAR, EMR, EHR or database in the cloud.

Any operations of the various methods described herein can be used incombination with or separately from one another, depending on thedesired features and in consideration of constraints such as financial,space, material and manufacturing availability.

The above detailed description includes references to the accompanyingdrawings, which form a part of the detailed description. The drawingsshow, by way of illustration, specific embodiments in which theinvention can be practiced. These embodiments are also referred toherein as “examples.” Such examples can include elements in addition tothose shown or described. However, the present inventors alsocontemplate examples in which only those elements shown or described areprovided. Moreover, the present inventors also contemplate examplesusing any combination or permutation of those elements shown ordescribed (or one or more aspects thereof), either with respect to aparticular example (or one or more aspects thereof), or with respect toother examples (or one or more aspects thereof) shown or describedherein.

In this document, the terms “a” or “an” are used, as is common in patentdocuments, to include one or more than one, independent of any otherinstances or usages of “at least one” or “one or more.” In thisdocument, the term “or” is used to refer to a nonexclusive or, such that“A or B” includes “A but not B,” “B but not A,” and “A and B,” unlessotherwise indicated. In this document, the terms “including” and “inwhich” are used as the plain-English equivalents of the respective terms“comprising” and “wherein.” Also, in the following claims, the terms“including” and “comprising” are open-ended, that is, a system, device,article, composition, formulation, or process that includes elements inaddition to those listed after such a term in a claim are still deemedto fall within the scope of that claim. Moreover, in the followingclaims, the terms “first,” “second,” and “third,” etc. are used merelyas labels, and are not intended to impose numerical requirements ontheir objects. The terms approximately, about or substantially aredefined herein as being within 10% of the stated value or arrangement.

The above description is intended to be illustrative, and notrestrictive. For example, the above-described examples (or one or moreaspects thereof) may be used in combination with each other. Otherexamples can be used, such as by one of ordinary skill in the art uponreviewing the above description. The Abstract is provided to allow thereader to quickly ascertain the nature of the technical disclosure. Itis submitted with the understanding that it will not be used tointerpret or limit the scope or meaning of the claims. Also, in theabove Detailed Description, various features may be grouped together tostreamline the disclosure. This should not be interpreted as intendingthat an unclaimed disclosed feature is essential to any claim. Rather,inventive subject matter may lie in less than all features of aparticular disclosed example. Thus, the following claims are herebyincorporated into the Detailed Description as examples or examples, witheach claim standing on its own as a separate example, and it iscontemplated that such examples can be combined with each other invarious combinations or permutations. The scope of the invention shouldbe determined with reference to the appended claims, along with the fullscope of equivalents to which such claims are entitled.

NOTES AND VARIOUS EXAMPLES

Each of these non-limiting examples may stand on its own, or may becombined in various permutations or combinations with one or more of theother examples. The examples are supported by the preceding writtendescription as well as the drawings of this disclosure.

Example 1 is a system for intravenous (IV) medications to deliver amedication from a syringe, the system comprising: a provideridentification sensor configured to identify (e.g., sense) provideridentification information; an injection portal configured to receivethe syringe; one or more medication sensors configured to identifymedication identification information that is coupled to the receivedsyringe when the received syringe is located in the injection portal andconfigured to capture an image of the received syringe; one or moredisplays; one or more storage devices; and processing circuitry that isin electrical communication with the provider identification sensor, theone or more medication sensors, the one or more displays and the one ormore storage devices, wherein the processing circuitry is configured toreceive the provider identification information and to store theprovider identification to at least one of the one or more storagedevices, wherein the processing circuitry is configured to sendinstructions to at least one of the one or more displays to output avisual image or representation of the received syringe on the at leastone display, wherein the processing circuitry is configured to determinea volume of medication dispensed from the received syringe based on animage of the syringe captured by the one or more medication sensors, andwherein the processing circuitry is configured to at least one of: savethe volume of medication dispensed to the one or more storage devices orsend instructions to at least one of the one or more displays to outputthe volume of medication dispensed on the at least one display.

In Example 2, the subject matter of Example 1 includes, wherein theinjection portal further comprises: an injection port that is configuredto fluidly couple to IV tubing; and at least one orienting memberconfigured to guide the received syringe having a diameter that is afirst diameter of a plurality of different diameters, to mate with theinjection port.

In Example 3, the subject matter of Examples 1-2 includes, wherein atleast one of the one or more medications sensors is located to capturean image of an inside of the injection portal, and wherein theprocessing circuitry is configured to receive a captured image of thereceived syringe and to calculate the volume of medication dispensedfrom the received syringe from the captured image by determining aninternal diameter of the syringe and measuring a distance a plunger ofthe received syringe moves to calculate an injected volume.

In Example 4, the subject matter of Examples 1-3 includes, wherein atleast one of the one or more medication sensors is an RFID interrogator,and wherein the medication identification information is an RFID tag.

In Example 5, the subject matter of Examples 1-4 includes, wherein atleast one of the one or more medication sensors is configured to readthe medication identification information and transmit a medicationidentity to the processing circuitry.

In Example 6, the subject matter of Examples 1-5 includes, wherein atleast one of the one or more medication sensors is a machine visiondigital camera.

In Example 7, the subject matter of Examples 1-6 includes, wherein whenthe provider identification sensor is configured to read the provideridentification information and to generate provider identification data,and wherein the processing circuitry is configured to receive thegenerated provider identification information and to compare thegenerated provider identification information to withdrawing providerinformation, wherein the withdrawing provider information includes anidentity of the provider who withdrew the syringe from a vending source.

In Example 8, the subject matter of Example 7 includes, wherein theprovider identification sensor is a barcode reader, an RFIDinterrogator, a retinal scanner, a facial recognition scanner, or afingerprint reader.

In Example 9, the subject matter of Examples 1-8 includes, wherein theprocessing circuitry is configured to send instructions to at least oneof the one or more displays to output one or more of: a brand name of adrug, a generic name of a drug, a drug concentration, a dosage of adrug, a dosage delivered, a fluid flow rate, a fluid volume delivered, apatient allergy, an over-dosing alert, a drug allergy alert and a druginteraction alert.

In Example 10, the subject matter of Examples 1-9 includes, wherein theprocessing circuitry is configured to transmit dispensing information toone or more of: an electronic anesthetic record (EAR) and an electronicmedical record (EMR), to automatically record dispensing informationabout the medication dispensed from the received syringe to the EAR orthe EMR.

In Example 11, the subject matter of Examples 1-10 includes, aninjection port located in the injection portal that is configured tofluidly couple the received syringe to IV tubing; and at least one clampin electrical communication with the processing circuitry, wherein theprocessing circuitry is configured send an instruction to actuate the atleast one clamp positioned one or more of upstream or downstream fromthe injection port, and wherein the processing circuitry is configuredto send instructions to the at least one clamp to inhibit dispensing ofthe medication or an IV fluid when an adverse condition is determined bythe processing circuitry.

Example 12 is a system for intravenous (IV) medications to deliver amedication from a syringe, the system comprising: a provideridentification sensor configured to identify (e.g., sense) a provider;an injection portal configured to receive the syringe; one or moremedication sensors configured to identify medication information that iscoupled to the received syringe when the received syringe is located inthe injection portal, wherein at least one of the one or more medicationsensors is located to capture an image of an inside of the injectionportal; one or more displays; one or more storage devices; andprocessing circuitry that is in electrical communication with theprovider identification sensor, the one or more medication sensors, theone or more displays, and the one or more storage devices, wherein theprocessing circuitry is configured to receive the provideridentification information and to store the provider identification toat least one of the one or more storage devices, wherein the processingcircuitry is configured to send instructions to at least one of the oneor more displays to output a visual image or representation of thereceived syringe on the at least one display, wherein the processingcircuitry is configured to receive a captured image of the receivedsyringe and to calculate a volume of medication dispensed from thereceived syringe from the captured image by determining an internaldiameter of the syringe and measuring a distance a plunger of thereceived syringe moves to calculate an injected volume, and wherein theprocessing circuitry is configured to at least one of: save the volumeof medication dispensed to the one or more storage devices or sendinstructions to at least one of the one or more displays to output thevolume of medication dispensed on the at least one display.

In Example 13, the subject matter of Example 12 includes, wherein theinjection portal further comprises: an injection port that is configuredto fluidly couple to IV tubing; and at least one orienting memberconfigured to guide the received syringe having a diameter that is afirst diameter of a plurality of different diameters, to mate with theinjection port.

In Example 14, the subject matter of Examples 12-13 includes, wherein atleast one of the one or more medication sensors is an RFID interrogator,and wherein the medication identification information is an RFID tag.

In Example 15, the subject matter of Examples 12-14 includes, wherein atleast one of the one or more medication sensors is configured to readthe medication identification information and transmit a medicationidentity to the processing circuitry.

In Example 16, the subject matter of Examples 12-15 includes, whereinthe processing circuitry is configured to send instructions to at leastone of the one or more displays to output one or more of: a brand nameof a drug, a generic name of a drug, a drug concentration, a dosage of adrug, a dosage delivered, a fluid flow rate, a fluid volume delivered, apatient allergy, an over-dosing alert, a drug allergy alert and a druginteraction alert.

In Example 17, the subject matter of Examples 12-16 includes, whereinthe processing circuitry is configured to transmit dispensinginformation to one or more of: an electronic anesthetic record (EAR) andan electronic medical record (EMR), to automatically record dispensinginformation about the medication dispensed from the received syringe tothe EAR or the EMR.

In Example 18, the subject matter of Examples 12-17 includes, aninjection port located in the injection portal that is configured tofluidly couple the received syringe to IV tubing; and at least one clampin electrical communication with the processing circuitry, wherein theprocessing circuitry is configured send an instruction to actuate the atleast one clamp positioned one or more of upstream or downstream fromthe injection port, and wherein the processing circuitry is configuredto send instructions to the at least one clamp to inhibit dispensing ofthe medication or an IV fluid when an adverse condition is determined bythe processing circuitry.

Example 19 is a system for delivering intravenous (IV) fluids from an IVbag fluidly that is coupled to a drip chamber and IV tubing, the systemcomprising: an IV scale configured to receive and support the IV bag;one or more sensors, wherein at least one of the one or more sensors isconfigured to identify an IV fluid in the IV bag hanging on the IV scaleand wherein at least one of the one or more sensors is located adjacentto the received drip chamber and is configured to capture an image ofthe drip chamber; one or more displays; one or more storage devices; andprocessing circuitry that is in electrical communication with the IVscale, the one or more sensors, the one or more displays and the one ormore storage devices, wherein the processing circuitry is configured toreceive a medication identity and a captured image of the drip chamberfrom the one or more sensors, and wherein the processing circuitry isconfigured to determine a fluid flow rate by analyzing an image of fluiddrops falling in the drip chamber including determining a size of dropsand a number of drops per unit time; wherein the processing circuitry isconfigured to at least one of: save the fluid flow rate to one or morestorage devices or send instructions to at least one of the one or moredisplays to output the fluid flow rate on at least one of the one ormore displays.

In Example 20, the subject matter of Example 27 includes, wherein the IVscale includes a hanger configured to support the IV bag, and whereinthe IV scale is configured to measure a combined weight of the IV bag,the IV drip chamber, the IV tubing and the fluids in the IV bag, andwherein the processing circuitry is configured to determine a reductionin a measured combined weight over time to determine a weight of thefluid removed from the IV bag and to convert the measured combinedweight over time to a fluid flow rate and an infused fluid volume.

In Example 21, the subject matter of Examples 27-28 includes, a floatlocated in the IV drip chamber, wherein when the processing circuitrydetermines from the captured image that the fluid drops in the dripchamber cannot be distinguished from one another and that the float ismoving, the processing circuitry is configured to measure the fluid flowrate by determining a reduction of a combined weight of the IV bag, theIV drip chamber, the IV tubing and fluid in the IV bag over time.

In Example 22, the subject matter of Examples 19-22 includes, one ormore electromechanical clamps that is in electrical communication withthe processing circuitry, wherein the processing circuitry is configuredto determine from the captured image that no fluid meniscus is presentin the drip chamber, the processing circuitry sends an instruction tocause at least one of the one or more electromechanical clamps tocompress the IV tubing to inhibit fluid flow prior to air entering theIV tubing.

Example 23 is at least one machine-readable medium includinginstructions that, when executed by processing circuitry, cause theprocessing circuitry to perform operations to implement of any ofExamples 1-30.

Example 24 is an apparatus comprising means to implement of any ofExamples 1-23.

Example 25 is a system to implement of any of Examples 1-23.

Example 26 is a method to implement of any of Examples 1-23.

What is claimed is:
 1. An automated data consolidation module includinga module for housing electronic and electromechanical medical equipmentand a portable digital camera, and a system to receive and record dataproduced by the electronic and electromechanical medical equipment andthe portable digital camera, the automated data consolidation modulecomprising: a housing configured to house electronic andelectromechanical medical equipment; a cowling that substantiallyconfines the electronic and electromechanical medical equipment; atleast one portable digital camera; and processing circuitry in wired orwireless electrical communication with the portable digital camera toreceive digital data produced by the portable digital camera; whereinthe digital data is automatically delivered to the processing circuitryand software, and the processing circuitry and software is configured tointerpret the digital data by: performing at least one of machinelearning (ML) and artificial intelligence (AI) analysis to identifyspecific visual elements of the image by matching the subject image toknown images stored in an image library, wherein the identified specificvisual elements of the image constitute dose events or response eventsand the processing circuitry is configured to add time stamps or otherindicators of time so that unrelated data can be temporally correlatedduring subsequent “big data” or clinical decision support analysis, andautomatically save the information provided by the matched specificvisual elements of the image or the image itself to an electronic recordor database.
 2. The automated data consolidation module of claim 1,wherein the portable digital camera, processing circuitry and softwareis configured to photograph, analyze and record at least one of a QRcode, a barcode or other coding technology that is attached to a medicalitem or device.
 3. The automated data consolidation module of claim 2,wherein the QR code, barcode or other coding technology that is attachedto a medical item or device identifies at least one of the type of itemor device, the manufacturer of the item or device and the model of theitem or device, vastly narrows the ML and AI image search by directingthe search to a folder containing images of that specific make and modelof item or device.
 4. The automated data consolidation module of claim1, wherein the portable digital camera includes a pistol grip to assistin holding and aiming the camera at the subject item or device.
 5. Theautomated data consolidation module of claim 1, wherein the portabledigital camera includes a laser pointer to assist in aiming the cameraat the subject item or device.
 6. The automated data consolidationmodule of claim 1, wherein the portable digital camera includes a liquidcrystal display (LCD) projector, a laser projector or cluster of lasersthat can proj ect an image such as a splattering paintball onto an itemor device after an image of the item or device is successfully acquiredby the portable digital camera.
 7. The automated data consolidationmodule of claim 2, wherein the QR code, barcode or other codingtechnology is attached to a subject medical item or device in aprescribed location on the medical item or device relative to thespecific visual elements to be identified and the location of the QRcode, barcode or other coding technology becomes a target for the laserpointer so that the scene is standardized and the portable camera isoriented to match the similarly structured scenes and cameraorientations of the known images stored in an image library.
 8. Theautomated data consolidation module of claim 2, wherein the QR code,barcode or other coding technology identifies the area or areas ofinterest within the scene so that the ML or AI functions can focus onthe specific area or areas of interest on the item or device that showthe desired information.
 9. The automated data consolidation module ofclaim 1, wherein the handheld digital camera, processing circuitry andsoftware may include machine vision capabilities for one or more of:pill identification, pill counting or “observation” of and verificationof a patient taking their pills.
 10. An automated data consolidationmodule including a handheld digital camera comprising: a housingconfigured to house electronic and electromechanical medical equipment;and at least one handheld digital camera; and processing circuitry inwired or wireless electrical communication with the handheld digitalcamera to receive and record digital data produced by the handhelddigital camera; and the handheld digital camera, processing circuitryand software is configured to photograph, analyze and record at leastone of a QR code, a barcode or other coding technology that is attachedto a medical item or device, identifying at least one of the type ofitem or device, the manufacturer of the item or device and the model ofthe item or device; and the digital data from the handheld digitalcamera also includes an image of the item or device that isautomatically delivered to the processing circuitry, and the processingcircuitry and software is configured to perform at least one of machinelearning (ML) and artificial intelligence (AI) analysis to identifyspecific visual elements of the image by matching the subject image toknown images stored in an image library and automatically save theinformation provided by the matched specific visual elements of theimage or the image itself to an electronic record or database; whereinthe QR code, barcode or other coding technology that is attached to amedical item or device that identifies at least one of the type of itemor device, the manufacturer of the item or device and the model of theitem or device, vastly narrows the ML and AI image search by directingthe search to a folder containing images of that specific make and modelof item or device.
 11. The automated data consolidation module of claim10, wherein the portable digital camera includes a pistol grip to assistin holding and aiming the camera at the subject item or device.
 12. Theautomated data consolidation module of claim 10, wherein the handhelddigital camera includes a laser pointer to assist in aiming the cameraat the subject item or device.
 13. The automated data consolidationmodule of claim 10, wherein the handheld digital camera includes aliquid crystal display (LCD) projector, a laser projector or cluster oflasers that can project an image such as a splattering paintball onto anitem or device after an image of the item or device is successfullyacquired by the handheld digital camera.
 14. The automated dataconsolidation module of claim 10, wherein the QR code, barcode or othercoding technology is attached to a subject medical item or device in aprescribed location on the medical item or device relative to thespecific visual elements to be identified and the location of the QRcode, barcode or other coding technology becomes a target for the laserpointer so that the scene is standardized and the camera is oriented tomatch the similarly structured scenes and camera orientations of theknown images stored in the image library.
 15. The automated dataconsolidation module of claim 10, wherein the QR code, barcode or othercoding technology identifies the area or areas of interest within thescene so that the ML or AI functions can focus on the specific area orareas of interest on the item or device that show the desiredinformation.
 16. The automated data consolidation module of claim 10,wherein the identified specific visual elements of the image constitutedose events or response events and the processing circuitry isconfigured to add time stamps or other indicators of time so thatunrelated data can be temporally correlated during subsequent “big data”or clinical decision support analysis.
 17. The automated dataconsolidation module of claim 10, wherein the handheld digital camera,processing circuitry and software may include machine visioncapabilities for one or more of: pill identification, pill counting or“observation” of and verification of a patient taking their pills. 18.An automated data consolidation module including a handheld digitalcamera comprising: a housing configured to house electronic andelectromechanical medical equipment; and at least one handheld digitalcamera; and processing circuitry in wired or wireless electricalcommunication with the handheld digital camera to receive and recorddigital data produced by the handheld digital camera; and the handhelddigital camera, processing circuitry and software is configured tophotograph, analyze and record at least one of a QR code, a barcode orother coding technology that is attached to a medical item or device,identifying at least one of the type of item or device, the manufacturerof the item or device and the model of the item or device; and thedigital data from the handheld digital camera also includes an image ofthe item or device that is automatically delivered to the processingcircuitry, and the processing circuitry and software is configured toperform at least one of machine learning (ML) and artificialintelligence (AI) analysis to identify specific visual elements of theimage by matching the subject image to known images stored in an imagelibrary; wherein the QR code, barcode or other coding technologyidentifies the area or areas of interest within the scene so that the MLor AI functions can focus on the specific area or areas of interest onthe item or device that show the desired information and automaticallysave the information provided by the matched specific visual elements ofthe image or the image itself to an electronic record or database. 19.The automated data consolidation module of claim 18, wherein theportable digital camera includes a pistol grip to assist in holding andaiming the camera at the subject item or device.
 20. The automated dataconsolidation module of claim 18, wherein the portable digital cameraincludes a laser pointer to assist in aiming the camera at the subjectitem or device.
 21. The automated data consolidation module of claim 18,wherein the portable digital camera includes a liquid crystal display(LCD) projector, a laser projector or cluster of lasers that can projectan image such as a splattering paintball onto an item or device after animage of the item or device is successfully acquired by the portabledigital camera.
 22. The automated data consolidation module of claim 18,wherein the QR code, barcode or other coding technology is attached to asubject medical item or device in a prescribed location on the medicalitem or device relative to the specific visual elements to be identifiedand the location of the QR code, barcode or other coding technologybecomes a target for the laser pointer so that the scene is standardizedand the portable camera is oriented to match the similarly structuredscenes and camera orientations of the known images stored in the imagelibrary.
 23. The automated data consolidation module of claim 18,wherein the identified specific visual elements of the image constitutedose events or response events and the processing circuitry isconfigured to add time stamps or other indicators of time so thatunrelated data can be temporally correlated during subsequent “big data”analysis.
 24. The automated data consolidation module of claim 18,wherein the handheld digital camera, processing circuitry and softwaremay include machine vision capabilities for one or more of: pillidentification, pill counting or “observation” of and verification of apatient taking their pills.
 25. An automated data consolidation moduleincluding a handheld digital camera comprising: a housing configured tohouse electronic and electromechanical medical equipment; and at leastone handheld digital camera; and processing circuitry in wired orwireless electrical communication with the handheld digital camera toreceive and record digital data produced by the handheld digital camera;and the handheld digital camera, processing circuitry and software isconfigured to photograph, analyze and record at least one of a QR code,a barcode or other coding technology that is attached to a medical itemor device, identifying at least one of the type of item or device, themanufacturer of the item or device and the model of the item or device;and the digital data from the handheld digital camera also includes animage of the item or device that is automatically delivered to theprocessing circuitry, and the processing circuitry and software isconfigured to perform at least one of machine learning (ML) andartificial intelligence (AI) analysis to identify specific visualelements of the image by matching the subject image to known imagesstored in an image library; wherein the QR code, barcode or other codingtechnology is attached to a subject medical item or device in aprescribed location on the medical item or device relative to thespecific visual elements to be identified and the location of the QRcode, barcode or other coding technology becomes a target for a laserpointer so that the scene is standardized and the camera is oriented tomatch the similarly structured scenes and camera orientations of theknown images stored in the image library; and automatically save theinformation provided by the matched specific visual elements of theimage or the image itself to an electronic record or database.
 26. Theautomated data consolidation module of claim 25, wherein the portabledigital camera includes a pistol grip to assist in holding and aimingthe camera at the subject item or device.
 27. The automated dataconsolidation module of claim 25, wherein the QR code, barcode or othercoding technology identifies the area or areas of interest within thescene so that the ML or AI functions can focus on the specific area orareas of interest on the item or device that show the desiredinformation.
 28. The automated data consolidation module of claim 25,wherein the identified specific visual elements of the image constitutedose events or response events and the processing circuitry isconfigured to add time stamps or other indicators of time so thatunrelated data can be temporally correlated during subsequent “big data”or clinical decision support analysis.
 29. The automated dataconsolidation module of claim 25, wherein the handheld digital camera,processing circuitry and software may include machine visioncapabilities for one or more of: pill identification, pill counting or“observation” of and verification of a patient taking their pills. 30.The automated data consolidation module of claim 25, wherein theportable digital camera, processing circuitry and software may includemachine vision capabilities for measuring distance or movement where theprecise distance of the camera from a subject item or device is criticalfor accuracy and the portable digital camera includes two or more laserpointers mounted on opposite sides of the portable camera, separatedfrom each other by one or more inches and the pointers are aimed inwardto form a triangle with the two or more laser beams crossing each otherat a prescribed point or distance in front of the portable camera wherethey form a single dot on the item or device when the portable camera isat the precise distance from the item or device.